Protocol Overview

How a session actually moves bytes. Most apps never touch this layer — they use session.doc('control'), named media streams, and session.request. This page is the honest map of what rides where.

Transport negotiation

The SDK dials QUIC/WebTransport first and falls back to WebRTC. Negotiation self-gates on browser capability and server support; session.mediaTransport reports the winner ('webtransport' | 'webrtc' | null while undecided or where negotiation never ran). AppOptions.allowWebTransport: false forces the WebRTC path. Signaling and doc-sync ride a WebSocket alongside the media transport; the SDK reconnects, renegotiates onto rebound backends, and fails over regions for you (see Transport).

Control = a CRDT desired-state doc

There is no pause/resume/configure command protocol. Session control is a Yjs CRDT document: the browser writes desired state, the runtime observes it (and writes back), and sync is incremental binary updates multiplexed over the session WebSocket.

session.doc('control').set({ prompt: { text: 'a forest at dawn' }, paused: true })

control = ctx.doc("control")            # the same doc, runtime-side
if control.get("paused"):
    ...

Because it is a CRDT, state survives reconnects via merge — there is nothing to replay. Ephemeral per-frame input (keyboard/mouse/pose) rides the doc's awareness/presence layer instead of doc state — see useInputPresence — so it clears automatically when a peer disconnects.

Correlated request/response rides the docs

Classic "call and await a result" semantics ride the same doc machinery — no extra endpoint, reconnect-safe by construction:

• session.request(payload) — the client writes the payload under an auto-generated id into a session doc; the runtime writes the correlated response back; resolves with the result.
• session.requestStream(payload) — the request envelope carries this leg's consumer_id; the runtime opens a dedicated addressed llm-resp:<id> stream and emits incremental token deltas plus a terminal body to that consumer only (per-request isolation by transport).
• session.complete(payload) — the non-stream convenience over the same addressed stream; resolves with the terminal body.

Media and data streams

Named streams (session.stream(name) ↔ ctx.stream(name)) carry media tracks over the negotiated transport, plus a data lane: messages() subscribes this peer and yields payloads; emit(payload) produces (the first producer on a name claims the direction — a conflict rejects loudly with StreamDirectionConflict, never a silent fallback). Media tracks are recorded by default (the DVR); seek() replays within the retained window. Data-lane payloads are relayed in memory and are never recorded.