#8973·jan

feat(cli): show the prompt-cache hit rate in the TUI during a run

Author: thinhlpgCreated Sep 17, 2026Updated Sep 17, 2026

Problem

#8964 landed the counters and a /context readout, but not a hit rate. Three gaps remain between what the TUI shows today and the epic's acceptance criterion — "the live hit rate is visible to the user during a run."

1. There is no session aggregate

The TUI holds exactly two cache numbers, and both are per-turn:

src-tauri/src/core/cli/tui.rs:2100-2101

rust
turn_cached_tokens: u64,
turn_cache_write_tokens: u64,

They are zeroed on every turn — tui.rs:2637-2638, tui.rs:2667-2668, tui.rs:5389-5390 — and the rendered line says so:

src-tauri/src/core/cli/tui.rs:6112

rust
"Prompt cache (last request): {read} read ({read_pct:.0}% of prompt), {} written"

One request is a sample, not a rate. The number the epic is about — what fraction of this session's prompt tokens were served from cache — is never computed. A session that hits 95% for twenty turns and then thrashes shows the same last-request line as one that thrashed throughout.

2. It is hidden in exactly the case worth seeing

src-tauri/src/core/cli/tui.rs:4800-4801

rust
let cache_reported =
    reported && (snapshot.turn_cached_tokens > 0 || snapshot.turn_cache_write_tokens > 0);

cache_summary_line returns None when cache_reported is false (tui.rs:6098). So a provider that does report cache usage and honestly reports zero tokens read renders identically to a provider that reports nothing at all: the line vanishes.

  provider reports 90K read   →  "Prompt cache (last request): 90K read (75% of prompt)"
  provider reports 0 read     →  (nothing)          ← the alarm condition
  provider reports nothing    →  (nothing)          ← indistinguishable

A 0% hit rate is the single most expensive state the agent can be in, and it is the one state the UI refuses to draw. On a write-billing provider it is worse than silence: a prefix written every turn and never read is a 25% surcharge on the whole prompt, and the user sees no line at all.

context_view_hides_prompt_cache_when_not_reported (tui.rs:29988) pins the not-reported case. Nothing pins the zero-hit case, because today they are the same code path.

3. It is on-demand only

The readout exists solely inside /context. Nothing appears during a run. A user watching a long session burn money has to interrupt and type a slash command to find out — and per (2), may get a blank.

Proposed change

Accumulate per session, render a rate, and separate "zero" from "unknown."

  1. Add session-cumulative counters alongside the per-turn ones — cached, written, and total prompt tokens — that are not reset by reset_turn_usage. The rate is session_cached / session_prompt, not a mean of per-turn percentages.
  2. Split the gate. cache_reported should mean the route reports cache usage, not the route reported a non-zero number this turn. Ideally derived from #8963's reports_prompt_cache_usage, which does not exist yet; until it does, latch "this route has reported at least once this session" rather than testing the current turn's value.
  3. Render 0% as 0%, and a non-reporting route as not reported.
  4. Surface the session rate during a run, not only under /context — the status line is the obvious home.

Suggested shape for /context:

Prompt cache (session):      1.2M read (91% of prompt), 140K written
Prompt cache (last request): 90K read (75% of prompt)

and on a route that reports nothing:

Prompt cache: not reported by this provider

Acceptance criteria

  • Session-cumulative cache read / write / prompt token counters exist and survive turn boundaries.
  • /context shows a session hit rate as a share of session prompt tokens, alongside the existing last-request line.
  • A reporting route with zero cache reads renders 0%, and a test pins that it is visually distinct from the non-reporting case.
  • A non-reporting route renders not reported, never a fabricated 0% — and the existing context_view_hides_prompt_cache_when_not_reported behaviour is preserved or consciously replaced.
  • The live session hit rate is visible during a run without typing a slash command.
  • Resuming a session does not report a hit rate that silently excludes the pre-resume turns, or it states that it is scoped to this process.

Related

  • Closes the gap left by #8964, whose criterion "The TUI shows a per-session hit rate; a non-reporting route shows 'not reported' rather than 0%" was not met by #8967.
  • Depends on #8963 for a clean reports_prompt_cache_usage signal; workable without it, at the cost of a latch heuristic.
  • The same Some(0) / None conflation exists in the CLI JSON envelope — src-tauri/src/core/cli/run_report.rs:69-70 aggregates with unwrap_or(0). Worth fixing in the same pass.
  • Prior art: DeepSeek-Reasonix puts the live hit rate in the TUI; oh-my-pi splits cacheRead / cacheWrite cost per provider. oh-my-pi#11897 is the case for visibility — two prompt injectors caused an 80.2% overspend with 98.7% of input tokens written to cache and never read, which a visible read-share would have shown immediately.

Part of #8956.