RFC: A2UI bulk data by reference (avoid paying output-token rates to render server-held data)

TL;DR

To render a data surface (table, chart, roster), A2UI currently requires the model to emit the entire dataset inline as render_a2ui tool-call arguments. Path binding only references the model's own output data model — there's no way to bind a surface to data the agent already holds. So a 1,000-row table the agent fetched in milliseconds gets re-serialized as ~27K output tokens before the surface can render: large latency, output-token cost, and a redundant data hop.

This proposes an additive, opt-in path: the agent supplies bulk data out-of-band; the middleware injects it into the surface's data model via updateDataModel. The model emits only the component structure + a path reference — never the rows. Inline behavior is unchanged, so cross-SDK parity holds.

Problem

render_a2ui takes components + a dataModel. A List/Row binds children to a data-model path (e.g. /rows), but the rows live in dataModel.rows, inlined in the model's tool-call args (the middleware extracts them from streaming TOOL_CALL_ARGS). Bindings resolve only against that model-produced data model — nothing points at server-held data. So every displayed row must be generated by the LLM as output tokens, even when the agent already has the data.

Measured cost (deterministic emitter, no model variance)

Emitting the exact render_a2ui call for an N-row table:

Rows	render_a2ui args	≈ output tokens
10	1,350 B	~338
200	22,067 B	~5,517
1,000	109,630 B	~27,408

Output-token cost scales linearly with the dataset. At real generation rates this is the "tens of seconds to render a table I already have" that users hit. The data also makes a wasteful middle hop (agent → model output → surface) paid at output-token rates, with transcription-drift risk on large datasets.

Related transport cost (compounds this): render_a2ui data-model updates are emitted as cumulative per-row snapshots during streaming, so a K-row surface ships ~1+2+…+K = O(K²) wire bytes (megabytes for a large repeated-data surface). By-reference data delivered as a single updateDataModel sidesteps both the output-token cost and this O(K²) snapshot amplification.

Proposed design (additive, opt-in)

The middleware already understands updateDataModel ({ surfaceId, path, value }) — today only the model produces it. Let the agent produce it:

1. Agent declares data out-of-band — attach { surfaceId, path, value } via forwardedProps.a2uiData (mirrors the existing forwardedProps.a2uiAction the middleware already reads) and/or agent state.
2. Model emits structure only — render_a2ui with components (a List bound to /rows), omitting the rows from dataModel. Tool guidelines: "when data is provided by reference, bind to its path; don't inline it."
3. Middleware merges — emits the model's updateComponents plus an updateDataModel built from the agent payload. The renderer already resolves path bindings against the data model (same mechanism as form pre-fill) — no renderer change.

Result: render_a2ui args carry structure + a path reference (tens of tokens), not the dataset.

Why it's safe

• Additive: no a2uiData ⇒ identical to today (model inlines, middleware extracts). No existing test/integration changes.
• Parity-preserving: updateDataModel is already in the cross-SDK schema (TS/Python/.NET); we add a producer, not a new wire contract.
• No new render semantics: path-binding against the data model already works.

Scope

• spec: document the by-reference contract (no breaking change to the op shape).
• @ag-ui/a2ui-middleware: read agent-supplied data; emit merged updateDataModel; update RENDER_A2UI_TOOL_GUIDELINES.
• adapters: typed helper to attach a2uiData (Strands / LangGraph first).
• dojo + e2e: an a2uiByReference feature asserting structure-only args render a populated surface and the dataset is absent from model output.

Open questions

1. Channel: forwardedProps.a2uiData (least invasive, side-channel) vs. a first-class schema field (cleaner, bigger change)?
2. Large datasets: by-reference removes output-token cost but the payload still crosses the wire once — streaming/windowing is follow-up.
3. Validation: happy to build a middleware + Strands spike that re-measures against the table above (expect ~flat args as N grows) once there's buy-in on the approach + channel.

Sourced from real user reports of data-heavy A2UI surfaces. Looking for maintainer feedback on the approach and the channel choice before a spike.