perf(/api/session): six frontend callers omit msg_limit and pull the full transcript + full session tool_calls (10 s / 28 MB vs 7 ms / 80 KB bounded on a 5k-row session)

Summary

Six frontend code paths call GET /api/session?session_id=<sid> with no msg_limit (and no messages=0). On a long WebUI-native session that shape returns the entire display transcript plus the entire session-level tool_calls list, redacts all of it, and serializes it — ~10 s and a 28 MB body on the session below — while the bounded msg_limit=30 request against the same session is 7 ms / 80 KB. The uncapped shape is what fires on the paths that run most often on mobile: offline/bfcache recovery, stream_end settle, cancel sync.

The _MAX_MSG_LIMIT = 500 clamp in _parse_msg_limit() only applies when the client sends a limit; the no-limit path is documented as the intentional "full transcript for callers that need it" escape hatch, so it is uncapped by design. None of the six callers appear to actually need the full transcript.

This is the "which container is hot in redaction" discriminator requested in #7310, measured offline on exp-v0.52.291 against a real session. Related: #6491 (same symptom for foreign/CLI sessions — this one is WebUI-native), #6351, #7081.

Environment

• hermes-webui exp-v0.52.291 (master 71689c6c), Python 3.11, Linux 6.8, self-hosted, single user, Tailscale-only access from Android Chrome
• Session: WebUI-native, 5,003 display rows (state.db: 2,591 tool / 2,062 assistant / 359 user), transcript text ≈ 9.9 MB, sidecar 41 MB, previously manually compressed (anchor at row ~2139). Content redacted throughout; only sizes/timings below.

Access-log evidence (one day, one session)

Bare /api/session?session_id=<sid> — no msg_limit, no messages=0 — 16 hits, every one 5–16 s:

01:35:02 5466ms   01:39:58 5102ms   01:40:04 5737ms   02:05:57 6084ms
02:07:55 5997ms   02:14:35 5995ms   03:01:19 9089ms   03:44:41 16526ms
03:44:46 4812ms   03:48:55 5368ms   17:07:03 7751ms   18:03:21 7321ms
19:08:41 5025ms   19:13:33 6054ms   19:17:44 9257ms   20:42:19 5011ms

Note the pairs 6 s apart (01:39:58/01:40:04, 03:44:41/03:44:46): two recovery paths firing back-to-back on one reconnect. The msg_limit=30&expand_renderable=1 tail loads for the same session (31 that day) were all fast.

Matching slow-stage records:

[SLOW] session_id=<sid> get_session=4156.2ms model_resolve=0.0ms compact=1151.6ms redact=5071.0ms json_write=4008.0ms total=14386.9ms
[SLOW] session_id=<sid> get_session=41.4ms   model_resolve=38.9ms compact=657.0ms  redact=3886.8ms json_write=1460.6ms total=6084.7ms

Offline measurement of the route's stages (same session, isolated process)

Harness: get_session() → _merged_session_messages_for_display() → compact() | {"messages", "tool_calls"} → per-key redact_session_data() dispatch → json.dumps. Sizes are len(json.dumps(field)).

What enters redaction on the no-limit path, by top-level key:

So for #7310's decision tree: the hot frames are _redact_messages and _redact_tool_calls on the two transcript-shaped containers, not _copy_json_value on a nested compact-session field and not load_settings(). compression_anchor_details, context_engine_state, compression_recovery, gateway_routing_history are all 2 bytes here.

Whole-response cost, uncapped vs bounded, same session:

Redaction time scales linearly with bytes redacted (~2.4 s per 6 MB across all three rows), so this is not a pathological regex; it is just the volume.

Transient memory per uncapped request in this harness: +49 MB Session object, +36 MB redacted copy, +9 MB body (28 MB bytes) — peak 450 MB for one request. Two of these overlapping (the 6-second pairs above) on a ThreadingHTTPServer is ~1 GB of transient allocation, which on our 2.2 GB MemoryHigh cgroup was enough to push the process into mem_cgroup_handle_over_high throttling and a multi-hour silent stall (alive, 0 requests served, needed SIGKILL) — the symptom that led here.

Second finding: session-level tool_calls windowing is gated on msg_limit

In the route:

_windowed_messages = (load_messages and msg_limit is not None and (...))
...
if _windowed_messages:
    _session_tool_calls = _tool_calls_for_message_window(_session_tool_calls, _messages_offset, len(_truncated_msgs))

So any path that does not window messages also ships the whole session-level tool_calls list — 6.2 MB / 1 s on this session, and that list is a legacy structure the frontend only needs for rows lacking per-message tool_calls. It is a large fraction of the bounded-request cost whenever windowing is skipped.

The six callers (all static/, current master)

refreshSession and _restoreSettledSession are the ones firing repeatedly in the log; they run on exactly the flaky-network / background-tab conditions where a 10 s request is most likely to overlap with itself.

Proposed fix (two independent halves)

Server (closes the class):

1. Apply a default window when msg_limit is absent and messages=1: treat missing as msg_limit=_MAX_MSG_LIMIT (500), set _messages_truncated/_messages_offset as the existing clamp path already does, so the frontend's "Load earlier" gate works. Callers that genuinely need everything can page with msg_before, which already exists and is already uncapped in its own read path. If an explicit escape hatch is wanted, make it opt-in (msg_limit=all) rather than the default.
2. Decouple tool_calls windowing from msg_limit is not None: window session-level tool_calls to the returned message range whenever a message range is returned, including the no-limit case (where the range is the whole list, so behaviour is unchanged there, but the code path stops depending on the limit flag).

Client (removes the traffic): 3. Change refreshSession(), _restoreSettledSession(), cancel sync and the /compress preflight to the bounded shape (&messages=1&resolve_model=0&msg_limit=30&expand_renderable=1, or messages=0 where only metadata is needed) and preserve hasServerOlder from _messages_truncated. The /undo and outline paths can decide whether they truly need the escape hatch.

(1) alone would have turned every 5–16 s request in the log above into a sub-100 ms one on this install, without any client change.

What I could not verify

• I did not run py-spy against the live process (ptrace_scope=1, no root); the per-key timing is from an isolated process running the same functions on the same store. It agrees with the in-process [SLOW] redact times (3.9–5.1 s in the log vs 3.9 s for messages alone here).
• I have not confirmed which recovery caller produced each specific log line; the access log does not carry the client call site. The 6-second pairs are consistent with _recoverFromOfflineSoftly → refreshSession followed by stream reattach → _restoreSettledSession.
• Not tested on a foreign/CLI session (that is #6491's territory).

Happy to test a patch against this session — it is a convenient worst case (5k rows, 41 MB sidecar) and I can re-run the harness before/after.