Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Layer 0 spec — Module io_uring, Stage 3a: registration (fixed resources)

Technical specification — Version 1.0 (target kernel: Linux 6.12 LTS)

Position. Stage 3a specifies resource registration with the kernel (io_uring_register(2), no. 427): fixed descriptor tables, registered buffers, registered ring fd, eventfd, personality, io-wq pool tuning, NAPI, clock. Sub-module air-sys-syscall::io_uring::registration. This Stage unlocks the “fixed”/“direct” variants referenced from Stages 2a–2c. Reuses the core of Stage 1.

Scope: 21 register opcodes (master doc, axis C — Stage 3a). Legacy variants (REGISTER_BUFFERS/FILES/FILES_UPDATE) are dropped in favour of the *2-tagged variants (master doc §4).


1. Cross-cutting conventions for Stage 3a

  • Explicit registration (ADR-022 D4): Air never registers automatically. The application that wants the benefit (fewer address translations, direct FDs) requests it explicitly.
  • Ownership of registered resources: a registered resource (buffer, FD) must remain valid as long as it is registered. The types RegisteredBuffers/FixedFdTable own the resource; usage references (RegisteredBufferSlice, FixedSlot) are tied by lifetime, and are therefore unusable after unregistration (safety by construction).
  • Link with teardown S2: a ring cannot be destroyed while resources are still registered without releasing them; shutdown()/Drop cleanly unregisters (owned buffers are returned).
  • Tags and sparse: the *2 variants support sparse registration (IORING_RSRC_REGISTER_SPARSE: empty slots to be filled later) and tags (FEAT_RSRC_TAGS: a CQE notification signals when a replaced resource is no longer referenced).
  • Registered ring fd: if the ring has registered its own fd (Stage 1, REG_REG_RING), all register/enter calls use it transparently (USE_REGISTERED_RING).

2. Fixed descriptor table — FixedFdTable

#![allow(unused)]
fn main() {
pub struct FixedFdTable { /* owns the OwnedFd of filled slots */ }

impl FixedFdTable {
    /// REGISTER_FILES2 (13). `capacity` slots, sparse by default (empty slots).
    pub fn register(ring: &mut IoUring, capacity: NonZeroU32) -> Result<Self, Errno>;
    /// REGISTER_FILES2 with an initial set of FDs.
    pub fn register_with(ring: &mut IoUring, fds: Vec<OwnedFd>) -> Result<Self, Errno>;
    /// REGISTER_FILES_UPDATE2 (14): places/replaces an FD in a slot.
    pub fn set(&mut self, ring: &mut IoUring, slot: u32, fd: OwnedFd) -> Result<(), Errno>;
    /// Clears a slot (FD returned to the caller).
    pub fn clear(&mut self, ring: &mut IoUring, slot: u32) -> Result<Option<OwnedFd>, Errno>;
    /// REGISTER_FILE_ALLOC_RANGE (25): bounds the auto-allocation range.
    pub fn set_alloc_range(&mut self, ring: &mut IoUring, range: Range<u32>) -> Result<(), Errno>;
    /// UNREGISTER_FILES (3): unregisters everything, returns remaining FDs.
    pub fn unregister(self, ring: &mut IoUring) -> Result<Vec<OwnedFd>, Errno>;
    /// Borrows a slot for use in an operation (IOSQE_FIXED_FILE).
    pub fn slot(&self, slot: u32) -> Option<FixedSlot<'_>>;
}

/// Borrowed reference to a filled slot; tied to the table by lifetime.
pub struct FixedSlot<'t> { /* index + borrow */ }
}

2.1 What FixedFdTable unlocks

  • Fixed-FD operations: read/write/send/recv… accept a FixedSlot instead of a BorrowedFd (flag IOSQE_FIXED_FILE) → the kernel avoids FD resolution on every op.
  • “Direct descriptor” variants from Stages 2a/2b — the result is stored in a slot instead of being returned as an ordinary FD:
#![allow(unused)]
fn main() {
impl IoUring {
    pub fn submit_openat2_direct(&mut self, dirfd: DirFd, path: CString, how: OpenHow,
        slot: FixedSlotTarget) -> Result<SubmissionToken, Errno>;
    pub fn submit_accept_direct(&mut self, listener: BorrowedFd<'_>, flags: AcceptFlags,
        slot: FixedSlotTarget) -> Result<SubmissionToken, Errno>;
    pub fn submit_socket_direct(&mut self, domain: SocketDomain, ty: SocketType, protocol: i32,
        slot: FixedSlotTarget) -> Result<SubmissionToken, Errno>;
}
/// Target slot: a precise index, or auto-allocation (IORING_FILE_INDEX_ALLOC).
pub enum FixedSlotTarget { Index(u32), Alloc }
}
  • FixedSlotTarget::Alloc ⇒ the kernel picks a free slot (within the range set by set_alloc_range) and returns it in cqe->res; otherwise -ENFILE.
  • To convert back to an ordinary FD: submit_fixed_fd_install (Stage 2c).
  • Asynchronous update in the stream: submit_files_update (Stage 2c, op 20).

Air use case: services with a high connection rate (compositor, AirCom) keep their sockets as direct descriptors — measurable gain, and confinement (a direct FD is not visible as an ordinary process FD).


3. Registered buffers — RegisteredBuffers

#![allow(unused)]
fn main() {
pub struct RegisteredBuffers { /* owns the pinned buffers */ }

impl RegisteredBuffers {
    /// REGISTER_BUFFERS2 (15). Pins the buffers; sparse/tags supported.
    pub fn register(ring: &mut IoUring, buffers: Vec<Vec<u8>>) -> Result<Self, Errno>;
    /// Variant backed by owned MmapRegion instances (data plane).
    pub fn register_mmap(ring: &mut IoUring, regions: Vec<MmapRegion>) -> Result<Self, Errno>;
    /// REGISTER_BUFFERS_UPDATE (16): replaces a buffer (tag → notification when
    /// the old one is no longer referenced).
    pub fn update(&mut self, ring: &mut IoUring, index: u32, buffer: Vec<u8>) -> Result<(), Errno>;
    /// CLONE_BUFFERS (30): clones the registered buffers from another ring.
    pub fn clone_from(ring: &mut IoUring, src: &IoUring) -> Result<Self, Errno>;
    /// UNREGISTER_BUFFERS (1): returns the buffers.
    pub fn unregister(self, ring: &mut IoUring) -> Result<Vec<Vec<u8>>, Errno>;
    /// Slice of a registered buffer, for read_fixed/write_fixed (Stage 2a).
    pub fn slice(&self, index: u32, range: Range<usize>) -> Option<RegisteredBufferSlice<'_>>;
}

/// Reference to a slice of a registered buffer; tied by lifetime.
pub struct RegisteredBufferSlice<'b> { /* index + range + borrow */ }
}
  • Gain: pinning and address translation are done once at registration → read_fixed/write_fixed (Stage 2a) and send_zc/recv fixed (Stage 2b) avoid this per-op cost. Hot path of the AirCom data plane (register_mmap on a shared memfd).
  • clone_from: shares buffers between rings within the same process (thread-per-core) without re-pinning.
  • Ownership: RegisteredBufferSlice cannot outlive unregister (lifetime) — read_fixed on a stale slice does not compile.

4. Registered ring fd

#![allow(unused)]
fn main() {
impl IoUring {
    pub fn register_ring_fd(&mut self) -> Result<(), Errno>;     // RING_FDS (20)
    pub fn unregister_ring_fd(&mut self) -> Result<(), Errno>;   // UNREGISTER_RING_FDS (21)
}
}
  • Registers the ring’s fd in an internal table → subsequent io_uring_enter calls use IORING_ENTER_REGISTERED_RING (no FD resolution). Net gain on paths with frequent enter calls. Requires/activates FEAT_REG_REG_RING.
  • Also used to designate a target ring in msg_ring (Stage 2c) in a decoupled manner.

5. eventfd notification

#![allow(unused)]
fn main() {
impl IoUring {
    pub fn register_eventfd(&mut self, efd: BorrowedFd<'_>) -> Result<(), Errno>;        // EVENTFD (4)
    pub fn register_eventfd_async(&mut self, efd: BorrowedFd<'_>) -> Result<(), Errno>;  // EVENTFD_ASYNC (7)
    pub fn unregister_eventfd(&mut self) -> Result<(), Errno>;                           // UNREGISTER_EVENTFD (5)
}
}
  • Binds the ring to an eventfd (family ipc): the kernel writes to it on every posted completion → a reactor can wait for completions via the eventfd (epoll integration, or cross-thread wake-up).
  • register_eventfd_async only notifies for completions processed asynchronously (filters out the noise from inline completions). Temporary deactivation via the IORING_CQ_EVENTFD_DISABLED flag (CQ ring, Stage 1).

6. Personality (identities)

#![allow(unused)]
fn main() {
pub struct Personality(/* id */);

impl IoUring {
    /// REGISTER_PERSONALITY (9): registers the current credentials.
    pub fn register_personality(&mut self) -> Result<Personality, Errno>;
    /// UNREGISTER_PERSONALITY (10).
    pub fn unregister_personality(&mut self, p: Personality) -> Result<(), Errno>;
}
}
  • Registers the calling process’s credentials and returns an id; an operation can then execute with those credentials by placing the id in sqe->personality. Allows a privileged service to perform an operation on behalf of a less-privileged identity (and vice versa) in a controlled manner.
  • Security link: combined with restrictions (Stage 3f) and the entitlements model (ADR-010), this is a least-privilege building block — exposed here as a primitive; policy-level use belongs to the upper layers.

7. io-wq pool tuning

#![allow(unused)]
fn main() {
impl IoUring {
    /// IOWQ_AFF (17): CPU affinity of io-wq workers.
    pub fn set_work_queue_affinity(&mut self, cpus: &CpuSet) -> Result<(), Errno>;
    /// UNREGISTER_IOWQ_AFF (18).
    pub fn clear_work_queue_affinity(&mut self) -> Result<(), Errno>;
    /// IOWQ_MAX_WORKERS (19): caps/limits for bounded/unbounded workers.
    pub fn set_work_queue_max_workers(&mut self, bounded: u32, unbounded: u32)
        -> Result<WorkQueueWorkerLimits, Errno>;
}
}
  • The io-wq pool executes operations that cannot be done inline (e.g. blocking I/O). set_work_queue_max_workers caps the thread count (categories IO_WQ_BOUND/IO_WQ_UNBOUND) — important on modest hardware (Charter principle 4: control the footprint on Raspberry Pi 4).
  • CpuSet shared with family-system (affinities).

8. NAPI busy-poll

#![allow(unused)]
fn main() {
impl IoUring {
    pub fn register_napi(&mut self, cfg: NapiConfig) -> Result<NapiConfig, Errno>; // NAPI (27)
    pub fn unregister_napi(&mut self) -> Result<(), Errno>;                        // UNREGISTER_NAPI (28)
}
pub struct NapiConfig { /* busy_poll_to, prefer_busy_poll */ }
}
  • Enables network busy-polling (NAPI) to reduce latency on very high-throughput sockets, at the cost of CPU. Reserve for profiles where latency is paramount (measure before enabling, Principle 5).

9. Clock source

#![allow(unused)]
fn main() {
impl IoUring {
    pub fn register_clock(&mut self, clock: ClockSource) -> Result<(), Errno>; // CLOCK (29)
}
pub enum ClockSource { Monotonic, Boottime, Realtime }
}
  • Sets the clock used by the ring’s timeouts (consistent with the TimeoutFlags of Stage 2c and family-time).

10. Summary of register opcodes (Stage 3a)

#Register opFacade
1UNREGISTER_BUFFERSRegisteredBuffers::unregister
3UNREGISTER_FILESFixedFdTable::unregister
4REGISTER_EVENTFDregister_eventfd
5UNREGISTER_EVENTFDunregister_eventfd
7REGISTER_EVENTFD_ASYNCregister_eventfd_async
9REGISTER_PERSONALITYregister_personality
10UNREGISTER_PERSONALITYunregister_personality
13REGISTER_FILES2FixedFdTable::register(_with)
14REGISTER_FILES_UPDATE2FixedFdTable::set / clear
15REGISTER_BUFFERS2RegisteredBuffers::register(_mmap)
16REGISTER_BUFFERS_UPDATERegisteredBuffers::update
17REGISTER_IOWQ_AFFset_work_queue_affinity
18UNREGISTER_IOWQ_AFFclear_work_queue_affinity
19REGISTER_IOWQ_MAX_WORKERSset_work_queue_max_workers
20REGISTER_RING_FDSregister_ring_fd
21UNREGISTER_RING_FDSunregister_ring_fd
25REGISTER_FILE_ALLOC_RANGEFixedFdTable::set_alloc_range
27REGISTER_NAPIregister_napi
28UNREGISTER_NAPIunregister_napi
29REGISTER_CLOCKregister_clock
30REGISTER_CLONE_BUFFERSRegisteredBuffers::clone_from

Total: 21 register opcodes. (PROBE → Stage 1; SYNC_CANCEL → Stage 1/2c; RESTRICTIONS/ENABLE_RINGS → Stage 3f; PBUF_RING/PBUF_STATUS → Stage 3b.)


11. New / shared types

New: FixedFdTable, FixedSlot<'t>, FixedSlotTarget, RegisteredBuffers, RegisteredBufferSlice<'b>, Personality, WorkQueueWorkerLimits, NapiConfig, ClockSource. Shared: OpenHow, AcceptFlags, SocketDomain/Type (Stages 2a/2b), CpuSet (family-system), MmapRegion (family-mem).


12. Test strategy

  • Integration: register a FixedFdTable, perform an accept_direct/openat2_direct (slot Alloc then Index), read/write via FixedSlot, fixed_fd_install back to an ordinary FD; RegisteredBuffers + read_fixed/write_fixed round-trip; clone_from between two rings; register_eventfd then verify the notification; register_personality + op with personality; set_work_queue_max_workers (read returned values); register_napi; register_clock.
  • Safety (compile-fail + Miri): a RegisteredBufferSlice/FixedSlot must not outlive unregistration (trybuild compile-fail test); no FD/buffer leak on unregister or teardown S2.
  • Property-based: sparse registration + progressive fill, tag consistency (notification when the old buffer is no longer referenced).
  • Errors via simulator: ENFILE (Alloc slot full), EINVAL, EBUSY, EFAULT.
  • Coverage: 100% lines + branches.

13. Key decisions that emerged at Stage 3a

  1. Legacy variants dropped — only the *2 (tagged) variants are exposed (master doc §4); this simplifies the API and makes tagging benefits mandatory.
  2. Owned resources + lifetime-tied references — safety by construction: it is impossible to use an unregistered resource.
  3. FixedSlotTarget::{Index, Alloc} — auto-allocation is a typed enum, not the raw IORING_FILE_INDEX_ALLOC sentinel (~0U) exposed directly.
  4. register_mmap for the data plane — registered buffers backed by owned MmapRegion instances (shared AirCom memfd), reusing the liveness handle from family-mem.
  5. io-wq caps/limits exposed early — to control CPU footprint on modest hardware, in accordance with the Charter.

14. Next steps

Next spec: io-uring-3b-provided-en.md (ring-mapped provided buffers: PBUF_RING/PBUF_STATUS, incremental consumption, automatic buffer selection for multishot recv/read). Global English translation after French document validation.


Document license: MPL 2.0 Status: Technical specification for Stage 3a (registration) of the air-sys-syscall::io_uring module, target kernel 6.12 LTS.