#6636·omnigent

tool_result (function_call_output) items still persist full base64 image data — the source rows #4310 leaves behind, unopenable on the managed store

Author: nakaneshinCreated Sep 6, 2026Updated Sep 20, 2026
LabelsBughelp wantedcomp:servercomp:reprP1-hightriagedvalidated:reproducedcomp:harness-t1

Summary

#4310 stripped base64 image data from compaction snapshot items, and #3133/#3113 did the same for pasted-image message items. But the item that produces those images in agentic sessions — the function_call_output (tool_result) item — still persists full inline base64. So a screenshot read by a tool (Read of an image, an MCP server returning an image block, a computer-use screenshot) is stored verbatim in the conversation store, one copy per image, for the life of the conversation.

This is the mirror of #4310: that issue fixed the snapshot of the history, but the source rows in the history are still multi-MB.

Where it is (v0.11.0)

redact_binary_payloads is wired into exactly one place — CompactionData:

omnigent/entities/conversation.py:492   return redact_binary_payloads(value, _binary_payload_omitted)   # CompactionData only

The other item data classes registered in ITEM_TYPE_TO_DATA_CLS do not strip:

  • MessageData ("message")
  • FunctionCallOutputData ("function_call_output") ← tool results, where Read-image / MCP-image / computer-use screenshots land
  • NativeToolData

The claude-native forwarder already removed the duplicate base64 (the toolUseResult metadata copy, via _tool_use_result_payload_omitted), which halved the per-image cost — but the primary copy in the tool_result content block is retained by design because the model re-sees it. That retained copy is never converted to a reference, so it accumulates.

Impact differs sharply by backend

  • Self-hosted Postgres: steady DB bloat, same shape as #4310 — unbounded over deployment life. (SELECT type, pg_size_pretty(sum(octet_length(data))) FROM conversation_items WHERE data LIKE '%base64%' GROUP BY type will show function_call_output rows carrying it.)
  • Managed store (Databricks EntityStore): worse — this is the failure mode I actually hit. Once an image-heavy conversation crosses the store's read-row/byte quota, hydrating it on open fails and the conversation becomes unopenable (Conversation not found / An internal error occurred). It is size-deterministic and does not clear on reload. A deck-QA session that Read a handful of PNGs into the parent conversation reproduced this reliably.

Worth noting for triage: the underlying bloat (base64 in the transcript) happens on standalone Claude Code too — its local JSONL stores the same bytes — but there it is a flat-file read with no quota, so it only makes resume slower, never unopenable. The hard failure is specific to the quota-backed managed store, which is why this matters more than a pure disk-usage issue.

Expected

function_call_output (and any tool_result-shaped) items should get the same treatment the pasted-image path already gets: store the image by file_id reference, hydrate the base64 from the file store only when building model input — not persist it inline in the conversation row.

The reference-on-store pattern already exists for user attachments (omnigent/inner/native_attachments.py resolves file_id blocks), so this is extending an established seam rather than inventing one, and it keeps the "model re-sees the image" guarantee intact (hydrate at replay, store a ref).

A narrower version, if full reference-on-store is too invasive: strip inline base64 from a tool_result item once it has been superseded by a compaction that covers it — i.e. an image the model will not re-see. That mirrors #4310's scope and bounds the change to rows that are already dead weight.

Notes

  • Not a duplicate of #4310 (compaction snapshot) or #3133/#3113 (message items) — this is the function_call_output path, which none of them touch.
  • Not #2299 (resume-transcript flattening / replay token cost) — this is the storage side.
  • Like #4310, any fix likely only shrinks newly written rows; existing rows keep their size unless a backfill is added.
  • Confirmed against omnigent 0.11.0 (built 2026-08-25), harness claude-native, managed (Databricks-fronted) server.

Environment

  • omnigent 0.11.0, claude-native harness, workspace-hosted (Databricks-fronted) server, macOS.