RFC: Agent-chosen span folding (begin_span / fold_span) — fold an investigation to its conclusion in place, keep the artefacts

Version line

v2 — Go rewrite (1.x), main-v2 (active development)

What problem does this solve?

An agent's context fills with investigation detail that stops being useful the moment it has yielded a conclusion. This is not waste in the ordinary sense — every step was necessary while it was being taken — but afterwards the evidence is one sentence and the search that found it is dead weight.

Two concrete cases, from building the UI layer of a project I work on:

• A compaction swallowed a plan. After an automatic compaction, the shape of the approved plan survived but its wording did not. The task list was no longer readable, so the acceptance criteria for each step had to be reconstructed. Losing the conclusions would have been a fair trade; losing the plan was not a decision anyone made.
• A refactor cost ten tool calls and deserved one line. Changing a Rust field from a tuple to a struct listed ~25 call sites across four modules, fixed in five compile/fix rounds. The durable fact is "the compiler enumerated them; no hunting". The ten error listings were load-bearing for about four minutes each.

The asymmetry is the point: investigation is expensive to read and cheap to summarise, but only the agent knows when the summary is safe to take.

Proposed solution

Two calls the agent emits like any other tool call:

begin_span("why there is no Rv → core::Value conversion")
    … greps, file reads, dead ends, four failed hypotheses …
fold_span("No conversion exists. Rv and core Value are different enums in
           different crates; closures/thunks are runtime-only. cbpv.rs:131, :579.")

The harness replaces everything between the two with the conclusion, in place: collapsed and visible, the way a compaction summary is, but scoped to the span the agent chose.

What exists today, and why it doesn't cover this

The last is closest, and it is genuinely the same idea: a subagent's tool calls and reasoning never enter the parent's context, and only its conclusion returns. That is why running a read-only investigation as a subagent feels clean. But most investigations cannot go in a box — they need to run a build, watch a test fail, ask the user a question, or decide whether a design is acceptable. Those are the ones that accumulate.

Note that the anchor machinery already mostly exists: compaction produces a summary of a region and keeps the original, and compress-style tools already take an anchor as a first-class parameter. What is missing is (a) a way to name a span by something other than a user message — the agent's own marker call would do — and (b) an agent-driven trigger alongside the automatic threshold one.

Design points that make or break it

• The agent decides when. The right moment is when the detail stops paying — which is not a turn boundary and not a token threshold. It is the moment the conclusion is reachable.
• Keep the artefacts, fold the search. A diff, a commit hash, a test result and a probe's output are durable. The greps that found them are not.
• Reversibility. The original transcript should still exist outside the model's view, expandable for the user. A fold is a projection, not a deletion — and that framing matters for the failure mode below.
• Nesting. A fold inside a fold, for an investigation that has sub-questions. The inner conclusion becomes part of the outer span.
• The failure mode is folding too early. The agent can drop something it needs later — most damagingly a reason rather than a fact. The mitigation is not to prohibit early folds but to state the rule: a conclusion must carry the why, not just the what. Folding a fact is cheap; folding a rationale is how a decision becomes unexplainable in six months.
• Inspection. The user must be able to see that a fold happened and expand it. A fold the user cannot see is indistinguishable from a mistake.

Interaction with prompt caching — which inverts the obvious intuition

Folding rewrites part of the message list, so it interacts with the prefix-cache economics this project is built around. Worth knowing before the feature settles, because it changes when a fold is cheap.

DeepSeek's cache docs state that matching is prefix-only from token 0: "Only requests with identical prefixes (starting from the 0th token) will be considered duplicates. Partial matches in the middle of the input will not trigger a cache hit." So a fold invalidates the cache from the fold point onward — everything before it still hits. The cost of a fold is the size of the tail, not the size of the span folded away. (kv_cache, news0802 — verified against the docs directly.)

That inverts the obvious intuition: folding a big old investigation is the expensive case, and folding a fresh one is almost free — what you pay for is the context that already exists after the fold, at the moment you do it. Two consequences:

• Fold promptly. Immediately after an investigation concludes, the tail is short and the conclusion is sharp; a day of conversation later, the same fold costs far more and the conclusion is hazier. The context-hygiene rule and the caching rule agree here — both reward summarising at the point where the summary is known to be right.
• The cost is one-off and self-healing. After a fold, the new prefix persists once DeepSeek's common-prefix detection fires ("When the system detects a common prefix across multiple requests, it will persist that common prefix as an independent cache prefix unit"). So a fold costs one cache-miss over the tail, not a permanent loss. That is a fair trade for reclaiming context, and it fits the existing cache-aware gating design rather than fighting it.

Granularity is 64 tokens ("content less than 64 tokens will not be cached"), so a fold that saves less than that is invisible to the cache and in the noise.

This is general, not a DeepSeek quirk — OpenAI's caching guide says the same of its own compaction feature ("can prevent reuse from the first changed token onward"). Anthropic goes further and exposes up to 4 cache breakpoints that update independently, which is the interesting alternative: with breakpoints a fold boundary can be declared rather than inferred, so the pre-fold region keeps its own cache entry. If the provider interface ever allows it, that is the knob to reach for.

Where this could be more than plumbing

This is the same idea as rendering a value at two levels of detail. We built a UITree with modes: FullInfo renders everything a value has, Friendly renders what the current task needs, and the mode decides which — not the function that happened to run. A conversation is a value of exactly this kind. An investigation is a tree of tool calls, and:

• the full transcript is its FullInfo,
• the conclusion is its Friendly,
• the fold is the projection between them, chosen by the participant that knows the purpose.

Which frames the real open question: who may fold — the agent, the host, or both? FullInfo is the host's automatic rendering; Friendly is where the type's own code gets a say. Folding is the agent getting a say about its own transcript; a host-driven threshold fold is the automatic rendering. Both are legitimate, and they are the same two rungs.

Worth stating plainly: this is much easier to design in from the start than to retrofit. If folding is a projection over a value the harness already knows how to render at two levels of detail, it comes nearly free; if the transcript is carried as an opaque message list, summarisation gets bolted on later.

Prior art, in one line

Summarisation checkpoints are common (conversation-summary memory in agent frameworks; /compact-style commands in coding agents). What appears to be missing is agent-chosen, span-scoped folding with the artefacts retained — everyone has the all-or-nothing version.

Suggested first step

The machinery exists: compaction already produces a summary of a region and keeps the original. Widen the anchor from "user message" to "marker the agent set", and add an agent-driven trigger alongside the automatic threshold one. That is a smaller change than the feature sounds, and it is testable in isolation: does a fold at turn N keep cache hits for everything before the fold point, and does the next turn restore prefix stability once common-prefix detection fires?

Open questions

• Marker placement rules. Are begin_span/fold_span allowed mid-turn, only at tool-call boundaries, or only between turns? (Between-turn is the simplest and probably right to start.)
• What happens to an unclosed span? Auto-fold at turn end with a warning? Leave it open and let the next begin_span close it?
• Interaction with the existing threshold compaction. Should a fold reset the compaction budget accounting, or is the folded span simply credited back?
• Rendering. Collapsed-by-default like a compaction summary, with an expand keybind, or shown inline as a one-line marker?
• Guardrails. Should there be a minimum retained-detail rule (e.g. refuse a fold whose conclusion has no rationale, or is under N tokens), or is that the agent's judgement to get wrong and learn from?

Related

• #8118 — pre-request pruning on warm sessions invalidates the prefix cache (same economics, different trigger)
• #6697 — desktop never lets the model choose its own compaction
• #9082 — extract key points from an over-length session into a fresh conversation (manual, whole-session version of this)
• #9520 — surface the remaining context budget every turn and document auto-compaction
• #244 — "extract ContextManager — consolidate 4 ad-hoc compaction mechanisms" (closed; the consolidation this would add a rung to)