Docs & Control
The control doc — the session's state surface — plus the correlated calls that ride it
A session's app-facing data surface is deliberately small: named streams for media and data, and synced docs for state — with correlated calls riding the docs when you need them. Most apps only ever touch the control doc — through session.doc(key) — plus named media streams via session.stream(name).
The control doc is the surface
For latency-sensitive browser-to-runtime state — prompts, camera/keyboard input, app settings — write the control doc. It is a CRDT-like JSON document synced over the session transport and delivered to Python as ctx.doc("control").
// Browser → runtime
session.doc('control').set({
prompt: { text: 'a forest at dawn' },
strength: 0.6,
})
// React to runtime-side updates
session.doc('control').on('change', (snapshot) => {
console.log(snapshot)
})# Python runtime reads the same doc — live, inline in a pipeline stage
control = ctx.doc("control")
prompt = control.get("prompt.text", "a sunset").set(patch) deep-merges into the document. .get(path?) reads a dotted path. Every doc(key) syncs independently to the runtime — control is the conventional key for steering state. On the Python side, cheap config is a live doc read inline in the stage; structural change (a model swap, a resolution change) uses doc.bind to reconstruct the affected stage — the old pipeline.bind(doc, mapping) field-mapping is retired.
Update cadence: the control doc is for state, not per-frame input
The control doc is a CRDT synced over the transport — ideal for steering state that changes a few times per second (prompt, style, settings, a camera pose you nudge). Rapid .set() calls coalesce: the SDK batches and sends the merged latest state, so firing .set() faster than the sync interval does not flood the wire, but it also does not guarantee every intermediate value is delivered. For per-frame, latency-critical input at 30–60 Hz (continuous keyboard/gamepad/pose in a world-model loop), use awareness presence — useInputPresence in React, createInputPresencePublisher in core — which is ephemeral by design (a dropped peer's input clears automatically); for raw media, publish an outbound track via session.stream(name).attachVideo(...).
Correlated calls ride the doc
Occasionally you want "call and get a result" semantics on a warm session — and they are built in as sugar over the same doc machinery, not a separate lane: no REST endpoint, no per-call connection, no hand-rolled correlation. session.request rides a session doc (the client writes the payload under an auto-generated id, the runtime writes the correlated response back), so an in-flight call even survives a reconnect:
const result = await session.request({ op: 'tryon', garmentId }, { timeout: 30_000 })
// Streaming (OpenAI stream:true shape) and its non-stream convenience:
for await (const delta of session.requestStream({ messages })) append(delta)
const body = await session.complete({ messages })# Python runtime: serve requests off the doc-based request lane
rpc = ctx.requests()
@rpc.on_request("tryon")
def tryon(req):
return run_model(req.payload)
rpc.start() # observes until the session endsThis keeps the expensive model resident (pay the warm-up once) while every individual call is a fast doc round-trip. In React, useRequest / useCompletion / useChat wrap the same surface — see Hooks.
Data on named streams
Discrete payloads the runtime pushes (status updates, per-image metadata, tokens) ride the data lane of a named stream: ctx.stream(name).emit(payload) on Python, consumed in the browser with session.stream(name).messages() or useStreamMessages in React. The first producer on a name claims the direction; a conflict rejects loudly with StreamDirectionConflict — never a silent fallback.
Related:
- App reference —
session.doc,session.stream,session.request - Wire protocol — how docs, requests, and streams move bytes
- Transport — phases, reconnection, viewer attach