Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Spec layer 0 — Module io_uring, Stage 3c: linked operations

Technical specification — Version 1.0 (target kernel: Linux 6.12 LTS)

Position. Stage 3c specifies chains of linked operations: expressing that an operation only starts after the success (or completion) of the previous one, without a userspace round-trip between the two. Sub-module air-sys-syscall::io_uring::linked. Relies on the SQE flags IOSQE_IO_LINK / IOSQE_IO_HARDLINK (axis D) and integrates the LINK_TIMEOUT op (15, Stage 2c). Reuses the core of Stage 1. No register opcode.

⚠️ Correction to io-uring-overview.md. The old overview describes the soft/hard link semantics backwards. The correct semantics (verified against the io_uring_enter(2) man page) are fixed in §2 below. The overview will be corrected during its reconciliation (its inventory role is already covered by the master document).


1. Principle

Several SQEs submitted together can form a chain: each link (chain element) only starts after the completion of the previous one. Marking is done via a flag on the predecessor link:

  • IOSQE_IO_LINK on op A ⇒ op B (the next SQE) is linked to A.
  • The tail of the chain is the first SQE without a link flag.

Without a chain, SQEs submitted together may execute in an arbitrary order and in parallel. The chain enforces ordering and dependency — essential for, e.g., socket → connect → send or openat2 → read → close in a single submission.


The two link types differ in their error propagation, not in ordering (ordering is guaranteed in both cases):

If a link (chain element) completes with an error, the chain is severed: all remaining links that have not yet started are cancelled and completed with -ECANCELED.

Broad definition of “error”: io_uring treats any unexpected result as an error that severs the chain — including a read/recv short read (fewer bytes than requested). This is the classic pitfall: a chain read(N) → process is severed if the read returns fewer than N bytes. This must be prominently documented in the facade.

A hard link does not sever the chain on a completion error result: the following links execute regardless.

Nuances:

  • A hard link implies a soft link (ordering is still guaranteed).
  • It only protects against completion errors: if the submission of the parent fails, the chain is severed anyway.

2.3 Table

Ordering guaranteedCompletion error on a linkSubmission failure of parent
soft (IO_LINK)yessevers the chain (-ECANCELED on subsequent links); short read = errorsevers
hard (IO_HARDLINK)yesdoes not interrupt (subsequent links execute)severs

3. API: LinkedChainBuilder

#![allow(unused)]
fn main() {
impl IoUring {
    /// Starts building a linked chain.
    pub fn link_chain(&mut self) -> LinkedChainBuilder<'_>;
}

pub struct LinkedChainBuilder<'ring> { /* &mut IoUring in staging mode */ }

impl<'ring> LinkedChainBuilder<'ring> {
    /// First link. `op` calls a normal `submit_*`, held in staging
    /// instead of being published.
    pub fn first<F>(self, op: F) -> Result<Self, Errno>
        where F: FnOnce(&mut IoUring) -> Result<SubmissionToken, Errno>;

    /// **Soft-linked** link to the previous one (sets `IO_LINK` on the previous).
    pub fn then<F>(self, op: F) -> Result<Self, Errno>
        where F: FnOnce(&mut IoUring) -> Result<SubmissionToken, Errno>;

    /// **Hard-linked** link to the previous one (sets `IO_HARDLINK` on the previous).
    pub fn then_hard<F>(self, op: F) -> Result<Self, Errno>
        where F: FnOnce(&mut IoUring) -> Result<SubmissionToken, Errno>;

    /// Time-bounds the **previous link** via a `LINK_TIMEOUT`
    /// (op 15). Must immediately follow the op to be bounded.
    pub fn with_link_timeout(self, spec: TimeoutSpec, flags: TimeoutFlags)
        -> Result<Self, Errno>;

    /// Publishes the **entire chain, atomically**. Returns the tokens in order.
    pub fn submit(self) -> Result<ChainTokens, Errno>;
}

/// Tokens for the links in a chain, in submission order.
pub struct ChainTokens { /* Vec<SubmissionToken> */ }
}

3.1 Staging mode (reuses the 55 submit_* methods without duplication)

For the lifetime of the builder, the &mut IoUring is in staging mode: a submit_* called inside the closure reserves a slot S1 and writes an SQE (with the appropriate link flag set on the predecessor), without publishing the queue. Publication only happens at the final submit(), which advances the SQ queue a single time (§3.3). This avoids duplicating 55 prepare_* methods: the same submit_* surface serves both direct mode and chain mode (consistent with Principle 7 — usage is explicitly delimited by link_chain()).

3.2 Atomic slot reservation

submit() only publishes if all of the chain’s slots could be reserved (S1). If the slab cannot hold the entire chain, the builder returns Err(EBUSY) before writing anything — no partial chain.

3.3 Publication

All the chain’s SQEs are written, then the SQ queue is advanced by a single store release (§3.2 of Stage 1) → the kernel sees the complete chain at once, guaranteeing that links are respected.

3.4 Example

#![allow(unused)]
fn main() {
// socket → connect → send, in a single linked submission.
let tokens = ring.link_chain()
    .first(|r| r.submit_socket(SocketDomain::Inet, SocketType::Stream, 0))?
    .then(|r| r.submit_connect(sock, addr))?      // starts after the socket
    .then(|r| r.submit_send(sock, buf, MessageFlags::empty()))?
    .submit()?;
}

4. Completions of a chain

  • One completion per link, in order. Each link has its slot S1 and returns its buffer normally.
  • Cancelled link (soft link severed): completion with res == -ECANCELED; slot/buffer returned.
  • With IOSQE_CQE_SKIP_SUCCESS (Stage 1, SubmitOptions) on intermediate links, you can wait only for the final completion (intermediate successes do not post a CQE) — useful for a chain where only the final result matters.
  • ChainTokens allows correlating each completion to its link.

with_link_timeout(spec, flags) inserts a LINK_TIMEOUT that bounds the previous link:

  • if the timeout expires before the link finishes ⇒ the link is cancelled (-ECANCELED) and the chain is severed (the timeout itself completes);
  • if the link finishes before ⇒ the timeout is cancelled automatically.

This is the only safe way to emit a LINK_TIMEOUT in Air (emitting it as an isolated op is rejected, Stage 2c §3.3). Example: bounding a connect to 2 s.

#![allow(unused)]
fn main() {
let tokens = ring.link_chain()
    .first(|r| r.submit_connect(sock, addr))?
    .with_link_timeout(TimeoutSpec::after(Duration::from_secs(2)), TimeoutFlags::empty())?
    .submit()?;
}

6. Types added / shared

New: LinkedChainBuilder<'ring>, ChainTokens. Reuses: SubmissionToken, SubmitOptions (drain, skip-cqe), TimeoutSpec, TimeoutFlags (Stage 2c). No new kernel types.


7. Testing strategy

  • Integration: successful socket→connect→send chain (observed ordering); soft chain with an error in the middle ⇒ subsequent links -ECANCELED; short read severing a soft chain (explicit test of the pitfall); hard chain where an error does not interrupt; link_timeout expiring (link cancelled) and not expiring (timeout cancelled); skip_cqe_on_success on intermediates (only the final completion arrives).
  • Property-based: for any chain, order of completions = submission order; CQE count consistent with skips; no slot/buffer lost even on severance.
  • Atomicity: insufficient slab ⇒ EBUSY with no partial publication (no SQE published).
  • Safety: Miri on buffer return for cancelled links.
  • Coverage: 100% lines + branches.

8. Key decisions that emerged at Stage 3c

  1. Correct semantics frozen — soft = severs on error (short read included); hard = does not sever on completion error. (Corrects the overview.)
  2. Staging mode on submit_* — no prepare_* duplication; usage explicitly delimited by link_chain().
  3. Atomic slot reservation + single publication — never a partial chain; EBUSY upfront if the slab is insufficient.
  4. link_timeout exclusively via the builder — isolated emission rejected (consistent with Stage 2c).
  5. skip_cqe_on_success for intermediates — wait only for the final result when that is the only useful one.

9. Next steps

Next spec: io-uring-3d-multishot-en.md (multishot operations: accept/recv/ poll/read multishot, MultishotToken token, CQE_F_MORE lifecycle, interaction with the provided buffers from Stage 3b). Global English translation after validation of the French documents.


Document license: MPL 2.0 Status: Technical specification for Stage 3c (linked operations) of the air-sys-syscall::io_uring module, target kernel 6.12 LTS.