Spec layer 0 — Module io_uring, Stage 3b: provided buffers (ring-mapped)

Technical specification — Version 1.0 (target kernel: Linux 6.12 LTS)

Position. Stage 3b specifies ring-mapped provided buffers (IORING_REGISTER_PBUF_RING), which replace the legacy op-based mechanism PROVIDE_BUFFERS/REMOVE_BUFFERS (retired, master doc §4). Sub-module air-sys-syscall::io_uring::provided. Reuses the core from Stage 1 and the registration from Stage 3a.

Scope: register opcodes PBUF_RING (22), UNREGISTER_PBUF_RING (23), PBUF_STATUS (26); submission flag IOSQE_BUFFER_SELECT; completion flags CQE_F_BUFFER and CQE_F_BUF_MORE.

1. The problem solved

For a classic recv, a buffer must be pre-committed per operation — therefore per pending connection. A server with 10,000 idle connections wastes 10,000 pinned buffers. Provided buffers invert this model:

The application registers a buffer group; it submits a recv/read with IOSQE_BUFFER_SELECT without an attached buffer; the kernel picks a buffer from the group when data arrives, and returns its identifier in the CQE (CQE_F_BUFFER, id in the upper 16 bits of cqe->flags).

Buffers are consumed only by connections that are actually active. Combined with multishot recv (Stage 3d), a single SQE serves an entire stream, with each datagram landing in a kernel-chosen buffer.

2. The buffer group — ProvidedBufferRing

#![allow(unused)]
fn main() {
pub struct ProvidedBufferRing { /* bgid, ring memory, backing buffers */ }

pub struct ProvidedBufferRingOptions {
    /// Ring memory allocated by the kernel (IOU_PBUF_RING_MMAP) then
    /// mmapped by the facade, instead of being supplied by the application.
    pub kernel_mmap: bool,
    /// Incremental consumption (IOU_PBUF_RING_INC): a buffer can serve
    /// multiple completions, consumed incrementally (see §5).
    pub incremental: bool,
}

impl ProvidedBufferRing {
    /// PBUF_RING (22). Registers a group `group_id` of `count` buffers of
    /// `buf_size` bytes (count = power of two). `count` also bounds
    /// the buffer ring.
    pub fn register(ring: &mut IoUring, group_id: u16, count: NonZeroU16,
                    buf_size: NonZeroU32, opts: ProvidedBufferRingOptions)
        -> Result<Self, Errno>;

    /// UNREGISTER_PBUF_RING (23). Returns the buffer memory.
    pub fn unregister(self, ring: &mut IoUring) -> Result<(), Errno>;

    /// PBUF_STATUS (26). Current head of the group (diagnostics / flow control).
    pub fn status(&self, ring: &IoUring) -> Result<ProvidedBufferRingStatus, Errno>;

    pub fn group_id(&self) -> u16;
    /// Number of buffers currently available (not checked out).
    pub fn available(&self) -> u16;
}

pub struct ProvidedBufferRingStatus { pub head: u32 }
}

• Two memory modes: buffers supplied by the application (default, on an owned MmapRegion — data plane); or ring allocated by the kernel (kernel_mmap) then mmapped at IORING_OFF_PBUF_RING | (group_id << IORING_OFF_PBUF_SHIFT).
• Ownership: ProvidedBufferRing owns the buffer memory for as long as the group is registered; unregister releases it. Teardown link S2: unregistration before the ring is destroyed.
• Replenishment: the application returns consumed buffers to the group (advances the tail of the buffer ring) — see §4.

3. Submission with automatic selection

#![allow(unused)]
fn main() {
impl IoUring {
    /// recv with automatic buffer selection from `group` (IOSQE_BUFFER_SELECT).
    pub fn submit_receive_provided(&mut self, sock: BorrowedFd<'_>, group: &ProvidedBufferRing, flags: MessageFlags)
        -> Result<SubmissionToken, Errno>;
    /// read with automatic buffer selection.
    pub fn submit_read_provided(&mut self, fd: BorrowedFd<'_>, group: &ProvidedBufferRing, length: u32, offset: Option<u64>)
        -> Result<SubmissionToken, Errno>;
}
}

• No buffer is attached to the submission: only the group_id is passed, along with IOSQE_BUFFER_SELECT. Slot S1 therefore does not hold a buffer (the buffer belongs to the group).
• Multishot variants (recv_multishot/read_multishot): Stage 3d — they rely on this same group.
• Bundle (IORING_RECVSEND_BUNDLE, FEAT_RECVSEND_BUNDLE): a single recv/send can consume multiple contiguous buffers from the group at once (referenced in Stage 2b) — exposed via a bundle: bool parameter here.
• Shortage/starvation: if the group is empty when data arrives, the completion carries -ENOBUFS; the application must replenish and resubmit.

4. Consumption: the ProvidedBuffer RAII guard

#![allow(unused)]
fn main() {
impl Completion {
    /// Retrieves the buffer chosen by the kernel for this completion.
    /// `None` if the completion did not use a provided buffer.
    pub fn into_provided_buffer<'r>(self, group: &'r mut ProvidedBufferRing)
        -> Option<ProvidedBuffer<'r>>;
}

/// RAII access to data received in a provided buffer. Returns the buffer to the
/// group (replenishment) when the guard is dropped.
pub struct ProvidedBuffer<'r> { /* id, useful length, borrow of the group */ }

impl<'r> ProvidedBuffer<'r> {
    pub fn id(&self) -> u16;
    pub fn data(&self) -> &[u8];          // length = bytes received (cqe->res)
    pub fn data_mut(&mut self) -> &mut [u8];
}
impl Drop for ProvidedBuffer<'_> { /* replenishes the group */ }
}

• Checkout → process → return cycle, made safe by RAII: the kernel has “checked out” a buffer (id in the CQE); into_provided_buffer materialises bounded access to the received bytes; dropping the guard returns the buffer to the group (advances the ring tail). Forgetting to return = buffer lost to the group → the guard returns it automatically.
• The &'r mut ProvidedBufferRing borrow guarantees that the group cannot be unregistered while a buffer is being processed.

5. Incremental consumption (incremental)

With ProvidedBufferRingOptions::incremental (IOU_PBUF_RING_INC):

• A large buffer can be consumed partially across multiple completions. The CQE then carries CQE_F_BUF_MORE (Completion::flags()), signalling that the same buffer will receive further completions and is therefore not returned automatically.
• The ProvidedBuffer guard reflects this: if CQE_F_BUF_MORE is set, dropping it does not replenish (the kernel continues using the buffer); replenishment only occurs on the final completion (without BUF_MORE). The application and the facade jointly track the consumption index (documented contract).
• Benefit: registering large ranges (a single large memfd) and consuming only what is needed per receive — memory-efficient for the AirCom data plane.

6. Summary

Replaces IORING_OP_PROVIDE_BUFFERS (31) and REMOVE_BUFFERS (32), retired to UNSUPPORTED.md.

7. Added / shared types

New: ProvidedBufferRing, ProvidedBufferRingOptions, ProvidedBufferRingStatus, ProvidedBuffer<'r>. New method on Completion: into_provided_buffer. Shared: MmapRegion (family-mem), MessageFlags (Stage 2b).

8. Test strategy

• Integration: register a group, recv_provided on a loopback socket, verify the returned buffer id and received bytes, drop the guard, verify replenishment (available() rises); buffer shortage (-ENOBUFS) followed by replenishment; kernel_mmap mode; multi-buffer bundle; status().
• Incremental: large buffer consumed across multiple recvs, sequence of CQE_F_BUF_MORE then final completion, replenishment only at the end.
• Safety (compile-fail + Miri): impossible to unregister the group while a ProvidedBuffer is live (mut borrow); no double return of a buffer; no read beyond the res received bytes.
• Property-based: for any checkout/return sequence, available ≤ count, no id returned twice, no buffer lost.
• Coverage 100% lines + branches.

9. Key decisions that emerged at Stage 3b

1. Ring-mapped mechanism only — the classic op-based approach is abandoned; the API is simpler and requires no syscall per buffer batch.
2. ProvidedBuffer RAII guard — replenishment is automatic on drop; it is impossible to “forget” to return a buffer.
3. &mut borrow of the group during processing — prevents unregistration from being called underneath a buffer that is being read.
4. Incremental consumption handled explicitly — CQE_F_BUF_MORE changes the return semantics of the guard; documented and tested, not left implicit.
5. MmapRegion backing — provided buffers for the data plane live on a shared owned memfd (reuses the liveness handle from family-mem).

10. Next steps

Next spec: io-uring-3c-linked-en.md (linked operation chains: soft/hard link, LinkedChainBuilder, integration of the link_timeout from Stage 2c).

Document license: MPL 2.0 Status: Technical specification for Stage 3b (ring-mapped provided buffers) of the air-sys-syscall::io_uring module, target kernel 6.12 LTS.