perf(agent): append system updates instead of replacing message 0

Problem

Whenever the system prompt changes, Jan deletes the old one and inserts the new one at index 0:

src-tauri/src/core/agent/upstream.rs:312-321

pub(crate) fn set_system_prompt(messages: &mut Vec<serde_json::Value>, system_prompt: &str) {
    messages.retain(|m| m.get("role").and_then(|r| r.as_str()) != Some("system"));
    messages.insert(
        0,
        serde_json::json!({ "role": "system", "content": system_prompt }),
    );
}

Called from the run setup at loop.rs:2064.

Replace-at-head is the single worst place to put a change. Message 0 is the first thing serialized, so any edit to it — one word in an assistant instruction, a plan-mode addendum appearing, a todo list updating — invalidates the whole prefix including the tool schemas behind it.

  replace-at-head (today)                    append-in-history (proposed)

  ┌──────────────────────────┐               ┌──────────────────────────┐
  │ system v2  ← rewritten   │  ✗ miss       │ system v1  ← untouched   │  ✓ hit
  │ tools                    │  ✗ miss       │ tools                    │  ✓ hit
  │ history                  │  ✗ miss       │ history                  │  ✓ hit
  │ user turn                │               │ system v2  ← appended    │  new bytes only
  └──────────────────────────┘               │ user turn                │
                                             └──────────────────────────┘
   whole prompt re-billed                      only the delta is new

A second, sharper hazard

retain removes every message with role: "system", not just the one it is about to replace. Compaction inserts its summary as a role: "system" message:

compaction.rs:78-106 — compact_conversation keeps the leading system messages, inserts the summary at sys_end as role: "system", then appends the kept tail.

So on any path where a compacted conversation is later reloaded and passed through set_system_prompt, the summary is a candidate for deletion. The headless CLI ignores MessagesUpdated outright — core/cli/mod.rs:1431-1433, "The non-interactive CLI doesn't persist session state, so MessagesUpdated is a no-op here" — so this is a desktop/TUI concern and is stated here as a risk to be tested, not a confirmed bug. Either way, a function that deletes by role is fragile when two features write that role for different reasons.

Proposed change

Append, don't replace.

  fn set_system_prompt(messages: &mut Vec<Value>, system_prompt: &str) {
-     messages.retain(|m| m.role != "system");
-     messages.insert(0, system_message(system_prompt));
+     // The head of the request is the cached region. A changed system prompt
+     // is appended as a new system node at the tail so the cached prefix —
+     // including the tool schemas behind it — survives the change.
+     if messages.first_system_content() == Some(system_prompt) { return }
+     if messages.is_empty() {
+         messages.insert(0, system_message(system_prompt));
+     } else {
+         messages.push(system_message(system_prompt));
+     }
  }

Two invariants fall out of this:

1. Never rewrite an already-sent node. Once bytes have gone on the wire, they are part of somebody's cache.
2. Delete by identity, not by role. If a node must be removed, remove the one that was written by the same producer — tag it — rather than everything sharing a role tag.

Models follow a later system message; this is how the majority of production agent loops apply prompt updates mid-session.

Acceptance criteria

• An unchanged system prompt produces zero mutations to the message array.
• A changed system prompt appends a new node; the bytes at every earlier index are identical to the previous request.
• set_system_prompt no longer removes messages by role alone; a test asserts a compaction summary survives a subsequent system-prompt update.
• A test asserts the model honours an appended system update (behaviour is preserved, not just bytes).
• The empty-conversation case still places the system prompt first.

Prior art

• DeepSeek Harness ships this exact mechanism as a named option — systemPromptUpdate: 'in-history'. A changed prompt becomes a new system node in the history rather than a swap at the head. It reports a 98.09% cache hit rate over 155.9M prompt tokens, and this is one of the two decisions that make that number reachable.
• OpenAI Codex goes further and gives every prefix item a content-derived UUIDv5 ID, so rewriting an already-sent item is detectable rather than silent.

Related

• Depends on #8957 and #8958 landing first — while volatile values are still merged into the system prompt, "the system prompt changed" is true on every turn and appending merely moves the cost instead of removing it.