[RFC] Working-memory / context-compaction increments for the lead agent

RFC: Working-memory / context-compaction increments for the lead agent

Summary

DeerFlow's long-term memory layer (facts, staleness review, consolidation, per-user buckets) is rich. Its runtime working-memory layer — context compaction — is a single tier: DeerFlowSummarizationMiddleware, an LLM summarizer that fires on a token/message/fraction trigger, replaces old messages with a summary stored in summary_text (projected into requests by DurableContextMiddleware), and keeps a recent window.

This RFC proposes four increments to that layer, ordered by how close each sits to a defect. Each is motivated on DeerFlow's own terms. This is a design discussion, and I'd like maintainer direction before opening any PR.

One hard constraint up front: DeerFlow is model-pluggable, so anything that assumes a known context window without a fallback is out of scope. All proposals stay model-agnostic.

Current behavior (verified)

• backend/packages/harness/deerflow/agents/middlewares/summarization_middleware.py extends LangChain SummarizationMiddleware.
• Shipped config.example.yaml: enabled: true, trigger: [{type: tokens, value: 32000}], keep: {type: messages, value: 10}. The bare schema default is enabled: false / trigger: null, so deployments that copy the example (the documented setup path) run with compaction on at 32k.
• The summary model is built with thinking_enabled=False. With the default model_name: null it resolves to config.models[0] — the first configured model — while the lead model is per-run selectable (configurable.model_name). The two coincide only when a run uses the default model.
• The fraction trigger silently no-ops when the model profile lacks max_input_tokens, and the profile it reads is the summary model's (_get_profile_limits → self.model.profile), not the current run's lead model.
• On summary-LLM failure, _summarize_with catches the exception and returns None: compaction is skipped for that turn and retried on every subsequent triggering turn.
• Tool output is size-capped at ingestion (ToolOutputBudgetMiddleware.wrap_tool_call) and re-budgeted in history at the model-call boundary (wrap_model_call → _patch_model_messages, request-only, checkpoint state untouched) — but always with the same static per-result budget; there is no staleness-based clearing. The heaviest tool-output work (deep-research subagents at max_turns=150) is quarantined in subagents, which summarize their own context.

Proposals

A — Bounded, observable failure handling for summarization

When the summary LLM fails persistently, _summarize_with returns None every triggering turn: the run pays one failing LLM call per turn, compaction never happens, and context is never bounded — silently, except for logs.

The failure mode is not config-dependent. With the default model_name: null the summary model is config.models[0], while the lead model is selected per run. A user running a long task on any non-default model while models[0]'s provider is broken (expired key, exhausted quota) sees every run succeed and compaction silently fail forever. No separately-configured summary model is needed to hit this.

• Directions to choose from: (1) startup/config-load validation of an explicitly configured summary model; (2) per-compaction fallback to the current run's model when the configured summary model fails; (3) a bounded-retry / breaker state so a persistently failing summary model stops adding a failed LLM call to every turn, with a visible warning.
• Aside (separately fixable, one line): the schema description says model_name: None means "use a lightweight model" (summarization_config.py), but the factory uses the default (first) model — the config.example.yaml comment ("null = use default model") is the accurate one.
• Question: which direction do you want — validation, fallback, breaker, or some combination?

B — Window-aware trigger semantics (the current OR-list cannot express them)

The default compacts at an absolute 32k tokens. On a 200k–1M-window model that begins lossy LLM summarization at ~3–16% window utilization, on nearly every long session — paying for summaries and dropping verbatim tool output the model could have used directly. 32k is a sound model-agnostic floor (DeerFlow ships no active model), but it is likely miscalibrated for the common 2026 setup of a large-context main model.

• The naive fix does not work, so this needs a new semantic: trigger evaluation is pure OR, so [{fraction: 0.8}, {tokens: 32000}] still fires at 32k on every model — the absolute trigger always wins. The current schema cannot express "absolute floor only when the profile exposes no window". A minimal semantic that can: an optional fallback on the fraction trigger, e.g. {type: fraction, value: 0.8, fallback_tokens: 32000}, where the fallback applies only when max_input_tokens is unavailable. Backward compatible; touches the trigger schema plus trigger evaluation.
• A second wrinkle any window-aware semantic must define: today fraction reads the summary model's profile, not the per-run lead model's. When a run selects a non-default model, the fraction is computed against the wrong window. "Window-aware" should mean the lead model's window.
• Evidence this layer needs a calibration/reliability pass: #3103 (64k-window model: fraction ineffective, a 51k tokens threshold reportedly not firing in time, runs dying on provider 400s) and #1602 (trigger misses in streaming mode without stream_usage). Those are under-triggering rather than early-triggering, but they point at the same trigger layer.
• Honest framing: the 32k default itself is a trade-off, not a defect — early compaction also bounds per-turn cost and can help models that degrade on long context.
• Question: is 32k deliberately conservative, and would a fraction-with-fallback semantic keyed on the lead model's window be welcome?

C — Two-stage summary prompt (reason in <analysis>, keep only <summary>)

Summarization is load-bearing: on by default, firing often at the 32k trigger, and its output replaces real conversation history. The summary model deliberately runs with thinking_enabled=False, so it has no native reasoning step. A two-stage prompt — a discarded <analysis> scratchpad, then the kept <summary> — restores a reasoning step at the prompt level: cheaper than enabling thinking, no reasoning-tag leakage, and the scratchpad tokens never enter the retained context. Prior art: Claude Code's publicly-known compact prompt uses exactly this analysis-first shape (supporting evidence; the argument stands on DeerFlow's own reliance on summarization).

• Precision: the current default (LangChain's DEFAULT_SUMMARY_PROMPT) is already structured multi-section (SESSION INTENT / SUMMARY / ARTIFACTS / NEXT STEPS) — what it lacks is a discarded stage: everything the model emits is retained verbatim into summary_text.
• This is a small code change, not config-only: _summarize_with stores response.text.strip() wholesale, so a two-stage prompt shipped only as a config example would put the <analysis> block into the summary — worse than today. The change: in _summarize_with / _asummarize_with, if a <summary> block is present keep only its content, else fall back to the full text (backward compatible; also covers manual /compact, which shares these methods). The opt-in prompt itself ships via the existing summarization.summary_prompt config field.
• Relationship to B: inverse coupling. If the trigger relaxes (B accepted), compaction is rarer and C matters less; if 32k stays deliberate, compaction is frequent and C matters more. C hedges B's rejection.
• Supporting evidence (beyond prior art): Let Me Speak Freely? (EMNLP 2024, arXiv:2408.02442) finds direct structured-format generation degrades reasoning — supporting the mechanism (a free-form reasoning stage before the structured output), though the study is on reasoning tasks, not summarization. Anthropic's own prompt-engineering guidance recommends <thinking>/<answer> structured CoT with answer-extraction — the exact shape here.
• Honest caveat: still a quality claim without a DeerFlow eval — proposed as opt-in, optionally accompanied by a small before/after eval; not a silent default change.
• Question: worth the extraction change + opt-in example (with or without eval data)?

D — A stale-tool-result tier before the summarizer — direction question, coordinating with #3568

DeerFlow is tool- and skill-heavy: multi-file sandbox reads, web_fetch / web_search, a fresh page snapshot per browser action, MCP outputs. Direct lead-agent tool use accumulates stale tool results in context until the LLM summarizer pays to compress them into prose.

• There is already an open PR in this space: #3568 (Headroom compaction middleware, stalled since June) — request-level, non-destructive compression of large tool outputs via the optional headroom-ai package. Before anyone (including me) writes more code here, the real question is direction: ML compression (#3568's approach), a simple deterministic staleness tier (clear old tool-result content to a short placeholder), or neither.
• The repo already has the right implementation shape for a deterministic tier: ToolOutputBudgetMiddleware.wrap_model_call → _patch_model_messages rewrites historical ToolMessages in the request only — checkpoint state untouched (SystemMessageCoalescingMiddleware follows the same pattern). A staleness tier as a deterministic request-level projection keeps tool_call_id chains intact by construction, and because the projection is deterministic, the rewritten prefix is stable across subsequent requests — one prefix-cache invalidation when a result first ages out, not churn every turn. It still trades against DeerFlow's deliberate prefix-cache-friendly prompt design, which is why this is a question, not a proposal to build.
• Scope note: subagent isolation already absorbs the heaviest case (delegated research summarizes inside the subagent), so this targets direct lead-agent tool use — real, but partially pre-absorbed.
• Question: what direction do you want for tool-output history — revive/land #3568, a deterministic staleness tier, or rely on subagent isolation + the summarizer by design?

Non-goals

• Nothing that assumes a known context window without an absolute fallback (model-pluggable constraint).
• No copying of other harnesses' window constants or provider-specific cache-edit mechanisms.
• No proactive-restructuring tier beyond the above.

---

RFC：Lead agent 的工作记忆 / 上下文压缩增量

摘要

DeerFlow 的长期记忆层（结构化 facts、staleness review、consolidation、按 user 分桶）相当完善。 但它运行时的工作记忆层——上下文压缩——只有一层：DeerFlowSummarizationMiddleware，一个 LLM 摘要器，按 token/message/fraction 触发，用摘要替换旧消息（摘要存进 summary_text，由 DurableContextMiddleware 投影进请求），保留最近一段窗口。

本 RFC 对这一层提四条增量，按「离缺陷有多近」排序。每一条都按 DeerFlow 自己的动机立。 这是一次设计讨论，在动手写任何 PR 前想先听 maintainer 的方向。

一个硬约束：DeerFlow 是模型可插拔的，任何「假设已知上下文窗口且无兜底」的方案都不在范围内。 下面所有提案都保持模型无关。

现状（已核实）

• backend/packages/harness/deerflow/agents/middlewares/summarization_middleware.py 继承 LangChain SummarizationMiddleware。
• 出厂 config.example.yaml：enabled: true、trigger: [{type: tokens, value: 32000}]、 keep: {type: messages, value: 10}。裸 schema 默认是 enabled: false / trigger: null，所以按官方 setup 拷 example 的部署，压缩是开着的、32k 触发。
• 摘要模型关掉 thinking 跑（thinking_enabled=False）。默认 model_name: null 时解析到 config.models[0]——配置里第一个模型——而 lead 模型是每 run 可选的 （configurable.model_name）。两者只在 run 用默认模型时才重合。
• fraction 触发在模型 profile 缺 max_input_tokens 时静默失效，且它读的是摘要模型的 profile （_get_profile_limits → self.model.profile），不是当前 run 的 lead 模型。
• 摘要 LLM 失败时 _summarize_with 捕获异常返回 None：本轮跳过压缩，之后每个触发轮重试。
• 工具输出在入口限大小（ToolOutputBudgetMiddleware.wrap_tool_call），并在 model-call 边界对 历史消息重套同一预算（wrap_model_call → _patch_model_messages，只改请求副本，checkpoint 状态不动）——但始终是同一个静态 per-result 预算，没有基于 staleness 的清理。最重的工具输出 负载（max_turns=150 的深度研究子代理）被隔离在子代理里、由子代理自己压缩。

提案

A — 摘要失败的有界、可观测处理

摘要 LLM 持续失败时，_summarize_with 每个触发轮返回 None：每轮多付一次失败的 LLM 调用、 压缩永远不发生、上下文永不 bound——除了日志之外完全静默。

这个失败模式不依赖特殊配置。默认 model_name: null 时摘要模型是 config.models[0]，而 lead 模型每 run 可选。用户用任何非默认模型跑长任务、而 models[0] 的 provider 恰好坏了（key 过期、配额 用尽）——每个 run 都成功，压缩永远静默失败。不需要单独配置摘要模型就能踩到。

• 可选方向：(1) 启动/加载配置时校验显式配置的摘要模型；(2) 摘要模型失败时该次压缩回退到当前 run 的模型；(3) 有界重试 / 熔断状态，让持续失败的摘要模型不再给每轮添一次失败调用，并给出可见 警告。
• **顺带（可单独修，一行）：**schema 里 model_name 的 description 写 "None = use a lightweight model"（summarization_config.py），但工厂实际用默认（第一个）模型——config.example.yaml 的注释（"null = use default model"）才是对的。
• **问题：**你们想要哪个方向——校验、回退、熔断，还是组合？

B — 窗口感知的触发语义（现有 OR 列表表达不了）

默认在绝对 32k token 压缩。对 200k–1M 窗口的模型，这意味着在 ~3–16% 窗口利用率就开始有损 LLM 摘要， 几乎每个长会话都命中——花钱做摘要、还丢掉模型本可直接使用的逐字工具输出。32k 是个合理的模型无关 下限（DeerFlow 出厂不带激活模型），但对当代「大窗口主模型」的常见配置很可能没调好。

• **天真的修法不成立，所以需要新语义：**触发判定是纯 OR，[{fraction: 0.8}, {tokens: 32000}] 在任何模型上仍然 32k 就触发——绝对阈值永远先命中。现有 schema 表达不了「仅当 profile 拿不到 窗口时才用绝对下限」。一个最小的可表达语义：给 fraction 触发加可选兜底，如 {type: fraction, value: 0.8, fallback_tokens: 32000}，兜底仅在 max_input_tokens 不可得时生效。 向后兼容；改动触发 schema + 触发判定。
• 任何窗口感知语义都得先定义的第二个问题：今天 fraction 读的是摘要模型的 profile，不是 每 run 的 lead 模型。run 选了非默认模型时，fraction 是按错误的窗口算的。「窗口感知」应当指 lead 模型的窗口。
• 这一层需要校准/可靠性整治的证据：#3103（64k 窗口模型：fraction 无效、51k 的 tokens 阈值 据报也没及时触发、run 死于 provider 400）与 #1602（streaming 下不开 stream_usage 时触发失灵）。 它们是触发不足而非触发过早，但指向同一个触发层。
• 诚实定性：32k 默认本身是权衡，不是缺陷——早压缩也能压住 per-turn 成本，还可能对「长上下 文会退化」的模型有利。
• **问题：**32k 是有意保守吗？一个锚在 lead 模型窗口上、带绝对兜底的 fraction 语义，你们要不要？

C — 双阶段摘要 prompt（在 <analysis> 里推理，只保留 <summary>）

摘要是承重件：默认就开、在 32k 触发下高频发生、而且它的输出会替换真实对话历史。摘要模型是 刻意关掉 thinking 跑的，没有原生推理步骤。双阶段 prompt——一个被丢弃的 <analysis> 草稿、再是 保留的 <summary>——在 prompt 层补回推理步骤：比开 thinking 便宜、没有 reasoning-tag 泄漏问题、 草稿 token 也不进保留上下文。先例：Claude Code 公开流传的 compact prompt 正是这个 analysis 先行 的形状（作为支持证据；主论证仍然立足 DeerFlow 自己对摘要的依赖度）。

• 措辞校准：现在的默认（LangChain DEFAULT_SUMMARY_PROMPT）已经是结构化多节的（SESSION INTENT / SUMMARY / ARTIFACTS / NEXT STEPS）——缺的是一个被丢弃的阶段：模型产出的所有内容都 原样进入 summary_text。
• 这是小代码改动，不是纯配置：_summarize_with 整段存 response.text.strip()，双阶段 prompt 若只作为配置示例提供，<analysis> 会进到摘要里——比现状更糟。改动是：在 _summarize_with / _asummarize_with 里，若存在 <summary> 块则只取其内容，否则整段回退（向后兼容；手动 /compact 共用这两个方法，一并覆盖）。opt-in prompt 本身走现有的 summarization.summary_prompt 配置字段。
• **与 B 的关系：**反向耦合。触发线放宽（B 被接受）→ 压缩变稀疏，C 边际价值下降；32k 是有意的 （B 被拒）→ 压缩高频，C 更重要。C 是 B 被拒时的对冲。
• 支持证据（超出先例）：《Let Me Speak Freely?》(EMNLP 2024, arXiv:2408.02442) 实证：直接产出 结构化格式会削弱推理——支撑本机制（在结构化输出前先有一个自由推理阶段），但该研究测的是推理 任务、不是摘要。Anthropic 官方 prompt 文档推荐 <thinking>/<answer> 结构化 CoT + 抽取 answer ——正是这里的形状。
• **诚实 caveat：**仍是没有 DeerFlow eval 的质量论断——按 opt-in 提议，可附一个小的 before/after 对比；不是静默改默认。
• **问题：**提取改动 + opt-in 示例 prompt（带或不带 eval 数据），值得吗？

D — 摘要器之前的旧-工具-结果档——方向问题，与 #3568 协调

DeerFlow 工具/skill 很重：沙盒多文件读取、web_fetch/web_search、浏览器每个动作回一张页面快照、 MCP 输出。直接在 lead agent 里用工具，旧工具结果会在上下文里堆着，直到 LLM 摘要器花钱把它们压成 散文。

• 这个问题空间已有一个 open PR：#3568（Headroom compaction middleware，六月起停滞）——请求级、 非破坏性地压缩大工具输出，走可选的 headroom-ai 包。在任何人（包括我）再写代码之前，真正的问题 是方向：ML 压缩（#3568 的路线）、简单的确定性 staleness 档（把旧工具结果内容清成短占位符）、 还是都不要。
• 确定性档的实现形状仓库里已有先例：ToolOutputBudgetMiddleware.wrap_model_call → _patch_model_messages 对历史 ToolMessage 的改写只发生在请求副本上——checkpoint 状态不动 （SystemMessageCoalescingMiddleware 同一模式）。staleness 档做成确定性的请求级投影： tool_call_id 链天然完整（状态从未被改），且投影是确定性的，改写后的前缀在后续请求间稳定—— 某条结果老化出局时前缀缓存失效一次，而不是每轮抖动。它仍要和 DeerFlow 刻意的前缀缓存友好设计 做权衡，所以这条是问题、不是「我要做」。
• **范围说明：**子代理隔离已经吸走了最重的场景（下放的研究在子代理里压缩），所以这条针对的是直接 在 lead agent 里用工具——真实，但已被预先吸收一部分。
• **问题：**工具输出历史你们想要哪个方向——把 #3568 救活合掉、一个确定性 staleness 档、还是认为 子代理隔离 + 摘要器在设计上已经够了？

非目标

• 任何「假设已知上下文窗口且无绝对兜底」的方案（模型可插拔约束）。
• 不抄其它 harness 的窗口常量或 provider 专属的 cache-edit 机制。
• 不做超出上述范围的主动重构档。