Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Spec layer 0 — Module io_uring, Stage 2c: async-specific operations

Technical specification — Version 1.0 (target kernel: Linux 6.12 LTS)

Position. Stage 2c specifies the 15 io_uring operations that have no direct filesystem/network equivalent: ring control, timers/timeouts, cancellation, event monitoring, synchronization, and inter-ring communication (master doc io-uring-0-inventaire-en.md, axis B). They reuse the core from Stage 1. Several form the foundation of APIs refined in Stage 3c (linked), 3d (multishot), or 3a (fixed resources) — cross-references are indicated.


1. Cross-cutting conventions for Stage 2c

  • poll32_events: only the 32-bit variant is exposed (PollEvents = bitflags on u32); the legacy poll_events field (u16) is dropped (master doc §4).
  • Bounded-wait timeouts vs TIMEOUT operations: bounded wait for a completion goes through wait_completion_timeout (EXT_ARG/ABS_TIMER, Stage 1 decision, §14.5). The TIMEOUT op in this Stage is for in-flow timers (standalone deadlines, completion counting, link-timeout) — explicitly distinct usages.
  • Shared memory (futex): a futex word lives in memory that must remain valid until completion; we reuse the liveness handle backed by MmapRegion (family-mem) introduced for madvise (Stage 2a §6.2).
  • Shared types: WaitTarget/WaitOptions/SignalInfo (family-process), EpollOp/EpollEvent (family-*), PollEvents. Refer to the family specs for equivalent syscall numbers.

2. Control: nop

#![allow(unused)]
fn main() {
pub fn submit_nop(&mut self) -> Result<SubmissionToken, Errno>;                 // OP 0
pub fn submit_nop_with_result(&mut self, injected: i32)
    -> Result<SubmissionToken, Errno>;                                          // NOP_INJECT_RESULT
}
  • Opcode: IORING_OP_NOP (0). No action; completes immediately.
  • Usage: ring priming/benchmarking, milestone in a chain (Stage 3c), testing (the _with_result variant injects a res via IORING_NOP_INJECT_RESULT — valuable for testing error paths without triggering a real kernel error).
  • Completion: completed() / into_result().

3. Timers/Timeouts

3.1 submit_timeout

#![allow(unused)]
fn main() {
pub fn submit_timeout(&mut self, spec: TimeoutSpec, flags: TimeoutFlags)
    -> Result<SubmissionToken, Errno>;                                          // OP 11
}
  • Opcode: IORING_OP_TIMEOUT (11).
  • TimeoutSpec: duration (__kernel_timespec) and/or a completion threshold (off = number of CQEs after which the timeout fires). A timeout can therefore be “after 10 ms” and/or “after N completions”.
  • TimeoutFlags: ABS (absolute deadline), BOOTTIME/REALTIME (clock selection), ETIME_SUCCESS (expiry is a success, not an error), MULTISHOT (repeated firings → multishot variant, Stage 3d).
  • Completion: completed(); res == -ETIME on expiry (or success if ETIME_SUCCESS).

3.2 submit_timeout_remove / submit_timeout_update

#![allow(unused)]
fn main() {
pub fn submit_timeout_remove(&mut self, target: SubmissionToken)
    -> Result<SubmissionToken, Errno>;                                          // OP 12
pub fn submit_timeout_update(&mut self, target: SubmissionToken, spec: TimeoutSpec, flags: TimeoutFlags)
    -> Result<SubmissionToken, Errno>;                                          // OP 12 + TIMEOUT_UPDATE
}
  • Opcode: IORING_OP_TIMEOUT_REMOVE (12); the update carries IORING_TIMEOUT_UPDATE. Cancels or re-arms an in-flight timeout.
  • Errors: ENOENT (unknown/already-fired target), EBUSY.
#![allow(unused)]
fn main() {
pub fn submit_link_timeout(&mut self, spec: TimeoutSpec, flags: TimeoutFlags)
    -> Result<SubmissionToken, Errno>;                                          // OP 15
}
  • Opcode: IORING_OP_LINK_TIMEOUT (15). Semantics: bounds in time the preceding operation in a linked chain; if the timeout expires first, the linked op is cancelled (ECANCELED), and vice versa.
  • Constraint: only meaningful when attached to an op via IOSQE_IO_LINK. The safe facade exposes it through the LinkedChainBuilder from Stage 3c (.with_link_timeout(spec)), not as a standalone op — emitting it alone returns an upstream validation error (Principle 4).

4. Cancellation: cancel (asynchronous)

#![allow(unused)]
fn main() {
pub fn submit_cancel(&mut self, target: CancelTarget, flags: CancelFlags)
    -> Result<SubmissionToken, Errno>;                                          // OP 14
}
  • Opcode: IORING_OP_ASYNC_CANCEL (14). Asynchronous variant of the sync_cancel from Stage 1 (which goes through register and is blocking).
  • CancelTarget/CancelFlags: by Token (user_data), by Fd, FdFixed, by Op, or Any (IORING_ASYNC_CANCEL_*). ALL = cancel all matching operations.
  • Completion: into_result() = number of cancelled operations; -ENOENT if nothing matches, -EALREADY if cancellation is already in progress.
  • Slab interaction: cancelling an op causes its completion to arrive with -ECANCELED; slot S1 (and its buffer) is returned normally. For multishot, the final completion (without F_MORE) releases the slot.

5. Event monitoring

5.1 submit_poll_add / submit_poll_remove / submit_poll_update

#![allow(unused)]
fn main() {
pub fn submit_poll_add(&mut self, fd: BorrowedFd<'_>, events: PollEvents)
    -> Result<SubmissionToken, Errno>;                                          // OP 6
pub fn submit_poll_remove(&mut self, target: SubmissionToken)
    -> Result<SubmissionToken, Errno>;                                          // OP 7
pub fn submit_poll_update(&mut self, target: SubmissionToken, events: PollEvents)
    -> Result<SubmissionToken, Errno>;                                          // OP 7 + POLL_UPDATE
}
  • Opcodes: POLL_ADD (6), POLL_REMOVE (7). Monitors an FD for events (read/write/error), like poll.
  • Completion (poll_add): into_poll_result() -> PollEvents (ready events). Single-shot here; multishot (IORING_POLL_ADD_MULTI, multiple notifications) and level-triggered (POLL_ADD_LEVEL) at Stage 3d.
  • poll_update: modifies the monitored events or the user_data of an in-flight poll (POLL_UPDATE_EVENTS/POLL_UPDATE_USER_DATA).

5.2 submit_epoll_ctl

#![allow(unused)]
fn main() {
pub fn submit_epoll_ctl(&mut self, epfd: BorrowedFd<'_>, op: EpollOp, fd: BorrowedFd<'_>, event: EpollEvent)
    -> Result<SubmissionToken, Errno>;                                          // OP 29
}
  • Opcode: IORING_OP_EPOLL_CTL (29). Equivalent: epoll_ctl (add/mod/del on an epoll set), executed asynchronously.
  • Completion: completed(). Note: in 6.12, this is the only io_uring epoll integration (EPOLL_WAIT only arrives after 6.12, out of scope). Useful for driving an existing epoll set without a synchronous syscall.

6. Synchronization

6.1 submit_futex_wait / submit_futex_wake / submit_futex_waitvimplemented

Implemented (coordinated family-mem PR, 2026-06-12). The historical &AtomicU32 signature (unsound in async: the borrow does not outlive the facade’s return) has been replaced by a reference into an MmapRegion + offset (see family-mem-mmap-region.md §3): the futex word lives in a shareable region kept alive by slot S1 (an MmapRegionLiveness), forbidding munmap while the op is in flight — no UAF, no leak (proven with Miri + loom).

#![allow(unused)]
fn main() {
pub fn submit_futex_wait(&mut self, region: &MmapRegion, offset: usize,
                         expected: u64, mask: u64, flags: FutexFlags)
    -> Result<SubmissionToken, Errno>;                                          // OP 51
pub fn submit_futex_wake(&mut self, region: &MmapRegion, offset: usize,
                         nr: u64, mask: u64, flags: FutexFlags)
    -> Result<SubmissionToken, Errno>;                                          // OP 52
pub fn submit_futex_waitv(&mut self, waiters: Vec<FutexWaiter>, flags: FutexFlags)
    -> Result<SubmissionToken, Errno>;                                          // OP 53

pub struct FutexWaiter {
    pub region: MmapRegion,   // owned (clone) ⇒ natural liveness guard
    pub offset: usize,        // bounded, 4-aligned (else EINVAL)
    pub expected: u64,        // no mask: the kernel futex_waitv carries none
}
}
  • Opcodes: FUTEX_WAIT (51), FUTEX_WAKE (52), FUTEX_WAITV (53).
  • Value: an io_uring reactor can wait on a futex without blocking its thread — integrating userspace synchronization into the same mechanism as I/O. futex_waitv waits on multiple futexes at once.
  • offset locates the (u32) futex word within the region: validated up front via MmapRegion::futex_word — region writable (WRITE, since the returned ref is mutable), bounds + 4-alignment — EINVAL otherwise, before submission (Principle 4). The FUTEX2_SIZE_U32 size is enforced (an MmapRegion word is an AtomicU32); only PRIVATE is exposed by FutexFlags. mask (for wait/wake) = futex bitset (addr3).
  • Safety: slot S1 retains the MmapRegionLiveness guard(s) (see §1) ⇒ the region stays mapped until completion.
  • ABI note: the kernel struct futex_waitv carries no per-waiter mask ({ val, uaddr, flags, __reserved }); FutexWaiter therefore exposes no mask (a silently-ignored field would be a footgun, ADR-032). For a masked wait, use submit_futex_wait (single-wait).
  • Completion: wait/waitv complete when woken up (or -EAGAIN if the value differs from expected at arm time) via completed(); wake returns the number of woken waiters via into_result().

6.2 submit_waitid

#![allow(unused)]
fn main() {
pub fn submit_waitid(&mut self, target: WaitTarget, options: WaitOptions, info: Box<SignalInfo>)
    -> Result<SubmissionToken, Errno>;                                          // OP 50
}
  • Opcode: IORING_OP_WAITID (50). Equivalent: waitid (see family-process.md; layer 0 convention: waitid preferred over wait/waitpid).
  • WaitTarget: pid / pgid / pidfd / all — typed (no raw integer).
  • Output: the kernel fills info (SignalInfo), owned buffer moved into the slot. Completion: into_waitid_result() -> Result<Box<SignalInfo>, Errno>.
  • Value: reaping a child process without blocking the reactor — a building block for a supervisor (relevant for air-launchd, layer 5).

7. Inter-ring and resources

7.1 submit_msg_ring

#![allow(unused)]
fn main() {
pub fn submit_message_ring_data(&mut self, target: &IoUring, res: i32, user_data: u64, flags: MessageRingFlags)
    -> Result<SubmissionToken, Errno>;                                          // OP 40 (MSG_DATA)
pub fn submit_message_ring_fd(&mut self, target: &IoUring, src_slot: u32, dst_slot: FixedSlotTarget, flags: MessageRingFlags)
    -> Result<SubmissionToken, Errno>;                                          // OP 40 (MSG_SEND_FD)
}
  • Opcode: IORING_OP_MSG_RING (40). Sends a message from one ring to another:
    • MSG_DATA: posts a CQE into the target ring (arbitrary res + user_data). A very lightweight inter-thread/inter-service notification mechanism — relevant for waking up a peer reactor (AirCom link).
    • MSG_SEND_FD: transfers a registered direct descriptor to another ring’s FD table, without going through sendmsg/socket.
  • MessageRingFlags: CQE_SKIP (no CQE on the target side), FLAGS_PASS (propagate flags to the target CQE).
  • Completion (sender side): completed(). Target side: a CQE appears in its CQ.
  • Safety: the target ring is borrowed (&IoUring); in practice a registered ring fd (Stage 3a) is often passed to decouple lifetimes — detailed in Stage 3a/3e.

7.2 submit_files_update

#![allow(unused)]
fn main() {
pub fn submit_files_update(&mut self, offset: u32, fds: Vec<RawFd>)
    -> Result<SubmissionToken, Errno>;                                          // OP 20
}
  • Opcode: IORING_OP_FILES_UPDATE (20). Updates slots in the registered FD table asynchronously (op equivalent of REGISTER_FILES_UPDATE2, but in the submission flow).
  • Cross-reference: the fixed FD table (FixedFdTable) is defined in Stage 3a.
  • Completion: into_result() = number of updated slots.

7.3 submit_fixed_fd_install

#![allow(unused)]
fn main() {
pub fn submit_fixed_fd_install(&mut self, fixed_slot: u32, flags: InstallFdFlags)
    -> Result<SubmissionToken, Errno>;                                          // OP 54
}
  • Opcode: IORING_OP_FIXED_FD_INSTALL (54). Converts a direct descriptor (which has no ordinary FD) into an ordinary FD, returned in the completion.
  • InstallFdFlags: NO_CLOEXEC (by default the installed FD is CLOEXEC).
  • Completion: completion.installed_fd() -> Result<OwnedFd, Errno>.
  • Usage: handing a real FD to third-party code from a direct descriptor obtained via direct accept/socket/openat2 (Stage 3a).

8. Summary of the 15 operations

# opcodeOpcodeFacadeCompletion
0NOPsubmit_nop / _with_resultcompleted / into_result
6POLL_ADDsubmit_poll_addinto_poll_result
7POLL_REMOVEsubmit_poll_remove / submit_poll_updatecompleted
11TIMEOUTsubmit_timeoutcompleted (-ETIME)
12TIMEOUT_REMOVEsubmit_timeout_remove / _updatecompleted
14ASYNC_CANCELsubmit_cancelinto_result
15LINK_TIMEOUT(via LinkedChainBuilder, Stage 3c)
20FILES_UPDATEsubmit_files_updateinto_result
29EPOLL_CTLsubmit_epoll_ctlcompleted
40MSG_RINGsubmit_message_ring_data / _fdcompleted
50WAITIDsubmit_waitidinto_waitid_result
51FUTEX_WAITsubmit_futex_waitcompleted
52FUTEX_WAKEsubmit_futex_wakeinto_result
53FUTEX_WAITVsubmit_futex_waitvcompleted
54FIXED_FD_INSTALLsubmit_fixed_fd_installinstalled_fd

9. Added / shared types

New in Stage 2c: TimeoutSpec, TimeoutFlags, PollEvents, CancelFlags, FutexFlags, FutexWaiter, MessageRingFlags, InstallFdFlags, FixedSlotTarget. Shared: WaitTarget, WaitOptions, SignalInfo (family-process), EpollOp, EpollEvent. New Completion methods: into_poll_result, into_waitid_result, installed_fd. FixedFdTable and the registered ring fd: Stage 3a.


10. Test strategy

  • Integration: nop (+ result injection), timeout (expiry + ETIME_SUCCESS + completion counting), cancel of an in-flight op (-ECANCELED received, slot returned), poll_add on a pipe then write, epoll_ctl add/mod/del, futex_wait woken by futex_wake, futex_waitv multi-wait, waitid on a terminating child, msg_ring data between two rings (CQE appears on the target side), fixed_fd_install of a direct fd.
  • Property-based: idempotent cancel (ENOENT/EALREADY), consistent poll events.
  • Safety: Miri/liveness handle for futexes (the futex word memory outlives the op); no FD leak on msg_ring_fd / fixed_fd_install.
  • Errors via simulator: ETIME, ECANCELED, ENOENT, EAGAIN (futex).
  • Coverage: 100% lines + branches.

11. Key decisions that emerged in Stage 2c

  1. link_timeout never as a standalone op — exposed only through the LinkedChainBuilder (Stage 3c); standalone emission is rejected upstream.
  2. async cancel vs sync_cancel — two deliberate paths: asynchronous in-flow (here) and synchronous blocking (Stage 1, basis for teardown S2).
  3. msg_ring recognized as an inter-reactor primitive — notification and FD transfer between rings; a potential building block for waking AirCom peers. Fine-grained inter-ring lifetime management is detailed in Stage 3a/3e.
  4. Async futex backed by the MmapRegion liveness handle — same memory safety mechanism as madvise (Stage 2a), no exposed unsafe.
  5. nop_with_result — exposed primarily for test tooling (covering error branches without a real kernel error).

12. Next steps

Next spec: io-uring-2d-cmd-en.md (URING_CMD: generic passthrough + socket commands SIOCINQ/SIOCOUTQ/get/setsockopt). Global English translation after validation of the French documents.


Document license: MPL 2.0 Status: Technical specification for Stage 2c (async-specific operations) of the air-sys-syscall::io_uring module, target kernel 6.12 LTS.