#990·plano

Pluggable cache-aware routing policies + eval harness

Author: adilhafeezCreated Jul 21, 2026Updated Jul 21, 2026
Labelsenhancementarea/routing

Follow-up from #982 review (raised by @knn-do).

The routing budget shipped in #982 is one specific cache-aware policy: stick to the warm anchor, gate paid switches by a cumulative overhead cap. The review raised that this policy is sensitive to the initial prompt and can hold a session on a sub-optimal route for an extended period, and asked for two things this issue tracks:

  1. Policy abstraction. Structure the session-routing decision (session_router::route) so alternative cache-aware policies can be implemented and selected via config — e.g. decaying stickiness, quality-delta thresholds, per-route budgets — without touching the decision plumbing (warmth inference, binding persistence, telemetry stay shared).
  2. Fast policy evaluation. A way to test candidate policies against recorded traffic and measure quality/cost tradeoffs systematically, instead of building docs/observability per policy. Building blocks already exist: record_counterfactual (the road not taken), plano.session.overhead_pct / baseline_in_usd / switch_spend_in_usd span attributes, and brightstaff_session_switch_decisions_total.

Open product question from the same thread (for @salmanap): maintain multiple strategies in Plano long-term, or evaluate and keep only the best one?