Agent template resends unbounded full chat/action history on every request
Problem
The agent starter currently has no bounded-history / context-budget layer.
On current main:
AgentChatManager.push()appends every prompt, continuation, and agent action to an unbounded array.ChatHistoryPartUtil.getPart()does astructuredClone(this.agent.chat.getHistory())of the entire history for every request.ChatHistoryPartDefinition.buildMessages()serializes every previous item back into model messages. For ordinary actions this includes the full action JSON.buildMessages()only sorts messages bypriority; it does not truncate, summarize, count tokens, or apply a context budget.
CHAT_HISTORY_PRIORITY = -Infinity therefore means "place history first", not "drop it first when the context fills".
A long-running canvas agent can grow request size monotonically even when old actions are fully reflected in the current canvas state.
Why this matters
The starter is explicitly designed for multi-step agentic loops. Canvas tasks can produce many low-level actions per user turn, so action history can dominate prompt size much faster than a normal chat transcript.
That has several effects:
- increasing latency / model cost on every subsequent step
- lower usable headroom for screenshots, visible shapes, context items, and user requests
- eventual provider context-window failures
- repeated transmission of stale geometry/actions that the current canvas representation already supersedes
Suggested direction
Add an explicit history-budget policy before model messages are built.
Possible shape:
- always preserve the current user request and a recent tail of history
- preserve user prompts / agent messages longer than low-level action traces
- compact older action runs into a bounded summary or drop them once their effects are represented by current canvas state
- budget by estimated tokens/serialized bytes, not only item count
- expose the policy as a starter-kit seam rather than hard-coding one provider's context size
The exact compaction strategy is open; the important invariant is that request size should not grow without bound with session length.
Acceptance criteria
- a long-running session has a deterministic maximum history budget
- recent user/assistant conversational context is retained
- old low-level action traces can be evicted or summarized
- current request / canvas context cannot be displaced by old history
- tests cover many-action sessions and verify bounded output size
- docs explain the default retention policy and how to customize it
Source: tldraw/tldraw