openclaw: update_mode=append lost on delta-retain fallback: session documents replaced by last retain window

Bug Description

When retaining with update_mode: "append" against a session-scoped document, the API's delta-retain path logs

Delta retain: no unchanged chunks for <doc_id>, falling back to full retain

and the fallback then stores only the incoming window instead of the prepended (existing + new) body. Each retain therefore replaces the session document with its most recent window, destroying the previously stored document text and the facts extracted from earlier windows. created_at still shows the original creation date, which masks the loss.

We hit this via the OpenClaw integration (plugin 0.12.0, retainEveryNTurns: 3, retainOverlapTurns: 1 → window = 4 turns): after three days of a long-running session, the document contained only the last ~4 turns of today, while retains had completed on all three days.

Evidence

Document openclaw:agent:main:dashboard:819560b4-… (bank unified-memory):

• created_at = 2026-09-13T20:11:07Z (first retain), updated_at = 2026-09-15T21:18:32Z
• document_metadata: retention_scope: "window", window_turns: "4"
• original_text begins at a message from 2026-09-15T20:43:35Z — everything before is gone
• all 52 chunks have created_at within 21:12–21:18 UTC on 2026-09-15 (re-created by the last op); none from the earlier retains
• memory units on the doc: 193, occurred dates only 2026-09-15 (135) or undated (58) — zero facts from 2026-09-13/14, although the operations list shows completed retains on 13.09 (3×), 14.09 (4×, one failed) and 15.09 (11×)

hindsight-api log (2026-09-15 19:18:37, op 39ac2c79-…, batch_retain):

hindsight_api.engine.retain.orchestrator - Delta retain: no unchanged chunks for
openclaw:agent:main:dashboard:819560b4-…, falling back to full retain

Plugin side is sending what the capability probe suggests: dist/index.js builds updateMode: useSessionScopedDoc ? "append" : undefined with the session-scoped document id (no per-turn suffix), and /version reports 0.9.2 with store_document_text: true (≥ 0.5.0, cf. #932).

Control experiment (append itself works)

Against the same API, on a test bank, single-item retains:

1. POST /v1/default/banks/<test>/memories with item {"content": "Alpha chunk …", "document_id": "paul-test:doc1", "update_mode": "replace"} → completed
2. same document, {"content": "Bravo chunk …", "update_mode": "append"} → completed
3. GET /documents/paul-test:doc1 → original_text contains both Alpha and Bravo.

So append works for a simple single-item retain. The failing plugin path differs in that it repeatedly appends windowed items (4-turn windows with context/metadata/window_turns) through the async batch_retain path on the same document — on that path the fallback appears to lose the prepended body.

Suspected location

hindsight_api/engine/retain/orchestrator.py:

• append handling fetches the existing original_text and prepends it (~lines 1615–1683, is_append = update_mode == "append" and … and is_first_batch)
• delta-retain check ~line 1756: _delta_full_body = document_body_override if update_mode != "append" else None, then _try_delta_retain(...)
• on "no unchanged chunks" it falls back to "full retain"
• final store ~line 2060: original_text=combined_content if config.store_document_text else None

Hypothesis: in the no-unchanged-chunks fallback, the body persisted is the incoming tail (document_body_override) rather than the prepend-combined content, so append semantics are lost exactly when the delta optimization bails.

Steps to Reproduce

1. hindsight-api 0.9.2 with store_document_text: true (docker compose, Postgres backend)
2. Retain repeatedly into the same document_id with update_mode: "append", sending windowed conversation chunks (4-turn windows, as the OpenClaw plugin does via the async batch_retain path), with content that shares no byte-identical chunks with what is stored (rolling conversation windows)
3. After ≥ 2 retains: GET /v1/default/banks/{bank}/documents/{doc_id}
4. Observe original_text/chunks/memory facts only cover the last window; the log shows Delta retain: no unchanged chunks …, falling back to full retain

Expected Behavior

With update_mode: "append" the document accumulates: existing text/facts are preserved and the new window is appended, regardless of whether the delta-retain optimization finds unchanged chunks.

Actual Behavior

The document is replaced by the last window's content (text, chunks and facts from earlier windows are gone). created_at keeps the original date, so the loss is not obvious in the UI.

Version

• hindsight-api: 0.9.2 (ghcr.io/vectorize-io/hindsight-api:latest-slim), docker compose with hindsight-control-plane and Postgres (tensorchord/vchord-suite:pg18-latest), store_document_text: true
• hindsight-openclaw (OpenClaw plugin): 0.12.0
• OpenClaw: 2026.9.4 (3a9d69d)

LLM Provider

Other (OpenRouter — extraction model openrouter/~z-ai/glm-flash-latest)

Related: #932 (append support), #4341 (session_end flush), #3686 (replace deletes memories on id replay — this report shows the append path on a stable session-scoped id also collapses to the last window).