test(agent): byte-level prefix regression tests

Problem

Jan's request-shape tests assert on the parsed object graph, not on the bytes that go on the wire:

src-tauri/src/core/agent/loop.rs:3722-3734

let request = build_completion_request("m", &messages, &[], &json!({}), None);
assert_eq!(
    request["messages"][0]["reasoning_content"], "the thinking",
    "reasoning must be resent unless the user opts out: {request}"
);

That assertion is correct and useful — and it cannot see any bug in this epic.

  what an object-graph assertion checks        what the provider checks
  ─────────────────────────────────────        ────────────────────────
  request["messages"][0]["role"] == "system"   the exact byte sequence of the
  request["tools"].len() == 14                 serialized request, from offset 0

     ✓ passes when the date changed               ✗ the cache misses
     ✓ passes when tools reordered                ✗ the cache misses
     ✓ passes when message 0 was replaced         ✗ the cache misses

Every failure in this epic is a byte-identity failure between two consecutive requests. An object-graph test compares one request against expectations. It never compares two requests against each other, and it never looks at ordering or serialization at all.

The comment directly above that test is telling — loop.rs:3720-3721 already says "dropping it shrinks earlier turns and forces the KV-cache prefix to be reprocessed". The concern is understood in the codebase. It is nowhere asserted.

This is why these bugs ship. oh-my-pi #7324 missed 11,239 of 20,968 requests (54%) in production; #11897 ran 14 hours at 2.9× cost with identical-looking output. Both would have passed every object-graph test in their suites.

Proposed change

Add a test module whose assertions are string comparisons on the serialized request.

prefix_stability
  common_prefix_len(a, b)                       # helper: bytes shared from offset 0

  stable_across_turns
    turn 1 → serialize → s1
    turn 2 (only user text differs) → s2
    assert s2.starts_with(prefix_of(s1))

  stable_across_midnight
    freeze clock 23:59:59 → s1
    advance clock to 00:00:01 → s2
    assert prefix(s1) == prefix(s2)

  stable_across_cwd_change
    project A → s1 ; project B → s2
    assert tools section of s1 == tools section of s2

  stable_across_restart
    build tool array in two separate processes
    assert serialized arrays are identical

  stable_across_mcp_timeout
    one server times out → s2
    assert the advertised array is unchanged from s1

  compaction_breaks_exactly_once
    run N turns past the compaction ratio
    assert exactly one prefix break occurred

Two properties make these tests worth the effort:

1. They fail on the thing that costs money, not on a proxy for it. A diverging byte offset is the defect, stated directly.
2. They report where. When one fails, print the common prefix length and a window around the first differing byte — the offset is usually enough to name the culprit without a debugger.

  FAILED prefix_stability::stable_across_midnight
    prefixes diverge at byte 17 of 4,182 (0.4% shared)

      expected  ...Today's date is 2026-09-16.\n\nYou are...
      actual    ...Today's date is 2026-09-17.\n\nYou are...
                                    ^

Add a guard test as well: assert the set of composers permitted to write above the cache line, so adding a new one fails the build until somebody decides where it goes. That is the mechanism #8966 formalizes at runtime; this is its compile-time half.

Acceptance criteria

• A prefix_stability test module exists whose assertions compare serialized request strings, not serde_json::Value graphs.
• Tests cover: consecutive turns, a midnight crossing, a cwd change, a simulated restart, an MCP listing timeout, and a compaction.
• A failing assertion reports the common prefix length and a window around the first differing byte.
• A guard test enumerates the components permitted to contribute to the prefix and fails when a new one appears.
• The tests run in CI on every PR touching the agent loop, the prompt composers, or the tool collection.
• Each test is documented with which epic item it protects.

Prior art

• OpenAI Codex ships codex-rs/core/tests/suite/prompt_caching.rs as a dedicated suite, and pairs it with responses_request_properties_match() — an exhaustive destructuring match, so adding a request field is a compile error until somebody decides whether it affects the cache. Type-level enforcement of exactly the guard described above.
• DeepSeek-Reasonix compares serialized bytes rather than parsed objects in its end-to-end tests.

Related

• Depends on #8962: a deterministic project() over a canonical transcript is what makes "serialize the same conversation twice and compare" a meaningful assertion.
• Compile-time counterpart to #8966.