RFC: opt-in RLM context engine (prompt-as-variable) to cut long-session tokens

Description

RFC: add an opt-in RLM context engine so long OMP sessions can keep large evidence outside the neural context window (prompt-as-a-variable), inspect it programmatically, and pay tokens only for selected slices / bounded subcalls.

This is not another compaction method. Today's ladder (remote → snapcompact → handoff → shake → soft in packages/coding-agent/src/session/compaction-methods.ts) is intentionally lossy at the provider-context layer: older turns are replaced so the next request fits. That stays the right default.

RLM (Zhang, Kraska, Khattab, arXiv:2512.24601) replaces llm.completion(huge_prompt) with rlm.completion(prompt) where the corpus is a REPL object. The root model peeks, slices, and optionally llm_querys a substring. The paper's useful result for OMP: the external environment helps even at depth 0 (no recursion); recursive subcalls help on information-dense tasks but are not uniformly cheaper.

Prime Intellect productized this as nano-rlm (ACP + persistent IPython) and prime-agent (arXiv:2608.23552), which forked pi / Oh My Pi and made the REPL the native tool. This RFC is the upstream-shaped slice: reuse OMP's existing retained Python kernel (EvalRunner / packages/coding-agent/src/eval/py) instead of forking the agent loop.

Lineage (already filed, do not re-open as Hermes work)

I previously wrote this up for Hermes, then closed the leaf into a research index. No equivalent OMP issue exists.

Related OMP issues (complementary, not duplicates)

Use Case

Long coding sessions where the expensive part is re-sending or re-summarizing evidence, not the current decision:

• repo / log / eval JSONL / session dump larger than the model window;
• tool results that shake drops and compaction then cannot cite;
• snapcompact image tokens when a 2k-token slice would have sufficed;
• multi-file reviews where compaction summaries lose cross-file relationships (the RLM paper's 10M-token setting).

Cost claim to validate, not assume: root turns stay O(working set); each llm_query bills only the slice + question; the corpus is not in the root prompt cache at all. Depth 0 should already reduce input tokens vs stuffing files. Depth 1 is opt-in and budgeted.

Area

Session management

Proposed Solution

Invariants

1. context.engine: native remains default. RLM is opt-in (context.engine: rlm and/or /rlm for one session).
2. Session JSONL stays authoritative. The RLM environment is derived, disposable, rebuildable from the journal + workspace files.
3. RLM and native compaction (and LCM if #6850 lands) are never composed on the same provider request. Unready / budget-exhausted / sandbox-down → fail open to the configured native method order.
4. No new always-on core tool. JSON tools stay the default agent loop. Do not make IPython the only tool (Prime Agent's fork; wrong for upstream OMP).
5. Hard mechanical budgets: max_depth (default 0), max_calls, max_total_tokens, max_cost, wall clock. Exhaustion is a clean stop with a partial trajectory, not a silent continue.
6. Sandbox: session Python kernel with no ambient secrets, no extra network, no arbitrary filesystem beyond the session cwd / explicit corpus bind. Host exec is not the production env.
7. Cancellation kills in-flight subcalls and leaves an honest JSONL trajectory.
8. Every answer span that came from the corpus cites byte/line/path; otherwise labeled model inference.

Why OMP already has the runtime hook

EvalRunner (packages/coding-agent/src/session/eval-runner.ts) already owns a retained per-session Python kernel (python.kernelMode, disposeKernelSessionsByOwner). pythonExecution messages can be excludeFromContext. That is depth-0 RLM without a second REPL stack:

• bind large artifacts as named objects in the kernel (corpus, files, journal slices);
• model-authored peeks return small strings that do enter the next root request;
• the full object stays in kernel memory / on disk, not in agent.state.messages.

Compaction of the root chat can still run; it must not delete the kernel or the bound corpus. Same as nano-rlm: kernel survives compaction.

v1 (depth 0) — ship this first

Paper: REPL-only already helps. Nano-rlm also truncates tool results >20KB to head/tail before they enter conversation.

1. Spill gate. When a tool result or file read exceeds a setting (start with 20KB, match nano-rlm), store it as a named env object and inject a stub + handle into the transcript (rlm://session/<id>/<handle>, size, hash, preview head/tail). Do not inject the full payload into the next provider request.
2. Peek tools (thin, not a new agent runtime): rlm_peek(handle, start, end), rlm_search(handle, regex, limit), implemented via the existing kernel. Results are size-capped.
3. llm_query(handle | text, question) — one bounded subcall through the existing model registry + usage accounting. Default max_depth=0 means this is a non-recursive LM call over a slice, not rlm.spawn.
4. Settings under Context (mirror #6850's context.engine split):

context:
  engine: native          # native | rlm   (lossless later if #6850)
  rlm:
    maxDepth: 0
    maxCalls: 32
    maxTotalTokens: 1000000
    spillBytes: 20480
    subModel: null        # default: active model

1. Slash: /rlm enable this session, /rlm status (kernel up, bound handles, calls/tokens/cost used, fail-open count), /rlm off.
2. Prompt cache: append a short runtime guide after the stable system prompt (do not rewrite it). Role-aware append only if depth ≥ 1 exists later.
3. Tests (offline): stub model + fixture corpus larger than window; assert full corpus bytes never appear in the serialized root request; assert peek/subcall citations; assert fail-open when kernel is down; assert /compact does not drop handles.

v2 (depth 1, only after v1 numbers)

Bounded subcall(slice, task) with max_depth: 1, supervisor-owned like nano-rlm (spawn waits for registration, not completion). Reuse OMP task subagents or kernel-side llm_query — pick one in implementation, do not grow a third agent runtime. Depth ≥ 2 is a non-goal until evals say otherwise.

Explicitly not this RFC

• Prime Agent /refine Continual Harness (Hermes #93306).
• Replacing bash/read/edit with Python.
• Training RLMs / verifiers / prime-rl.
• ACP ai.prime.rlm/runtime-v1 contract (nano-rlm training surface).
• Reconstructing a live kernel after process exit (session/load — nano-rlm also refuses this).
• Unrestricted host IPython.

Acceptance (v1)

• Corpus / spilled tool result larger than the model window can be processed without injecting it wholesale into the root prompt.
• Serialized provider request for the root turn never contains the spilled bytes (test fixture).
• Peek / llm_query results are size-capped and cited.
• Budgets are enforced; cancellation leaves a partial trajectory.
• Kernel down / budget hit → native compaction path, no hang.
• System prompt bytes unchanged for an existing session when RLM is toggled (cache rule).
• Bench vs native soft/shake/snapcompact on one repo-understanding fixture and one session-dump fixture: report input tokens, cost, and answer citations — not a quality-only vibe check.

Alternatives Considered

• Just add another compaction method. Compaction deletes evidence from the provider window. RLM never put it there. Different failure mode (lossy summary vs addressable store).
• Wait for #6850 LCM. LCM is a citation-bearing summary projection. Useful, still not original-byte peek. Exclusive routing can be shared; engines should not merge.
• Adopt Prime Agent as-is. They forked pi and made the REPL the only native tool. Wrong default for OMP's JSON-tool coding loop; too large a cutover.
• Implement on Hermes only (#93303). Closed, needs-decision, and OMP is where compaction cost actually lands for this workflow.
• Agent self-handoff (#1602). Complementary for task narrative; does not keep a 10MB log addressable.

References: RLM paper, alexzhang13/rlm, nano-rlm, prime-agent, Hermes #93303.