[Feature]: Jev router — use TypeSafe Jev (System One decision model) as a routing classifier with CEL `jev.*` variables

Prerequisites

• I have searched existing issues and discussions to avoid duplicates

Problem to solve

Bifrost's routing engine can already pick a model per request instead of per client: CEL Routing Rules plus the Complexity Router, which classifies each request into SIMPLE | MEDIUM | COMPLEX via an embedding nearest-phrase match with an optional LLM fallback classifier (plugins/routing/complexity/).

That classifier is the weak link for routing decisions:

• Embedding + nearest phrase only knows what the 150 reference phrases know. It cannot express "does this request need tools?", "is this a code edit or a question?", "is the user asking in Chinese?", or "which of my five candidate models fits" — it emits one of three fixed labels and a cosine similarity that is not a calibrated probability, so min_similarity is guesswork.
• LLM fallback (fallback: "llm", e.g. gpt-4o-mini) is slow (the docs budget 4 s), generates free text that has to be parsed, and costs a full chat call per routed request. It is why the fallback is opt-in and the default is none.

TypeSafe AI released Jev on 2026-09-15 (announcement, docs). It is a decision model, not a chat model: POST https://api.typesafe.ai/v1/systemone takes a state (text or JSON) and a map of typed questions, and returns typed answers with a probability distribution and a calibrated confidence — no text generation, no parsing:

// request
{
  "model": "jev-latest",
  "state": "<minimised request summary>",
  "questions": {
    "target":     {"type": "choice", "instructions": "Which model should serve this request",
                   "criteria": {"claude-fable-5-1": "...", "gpt-4o-mini": "...", "gemini-2.5-flash": "..."}},
    "needs_tools":{"type": "noul",   "instructions": "The request requires function calling"},
    "complexity": {"type": "score",  "instructions": "How hard is this to answer well",
                   "criteria": ["trivial lookup", "some reasoning", "deep multi-step reasoning"]}
  }
}
// response
{
  "model": "jev-1.13.0",
  "answers": {
    "target":      {"type": "choice", "choice": "gpt-4o-mini",
                    "probabilities": {"gpt-4o-mini": 0.81, "gemini-2.5-flash": 0.15, "claude-fable-5-1": 0.04},
                    "confidence": 0.62},
    "needs_tools": {"type": "noul",  "noul": 0.03},
    "complexity":  {"type": "score", "score": 0.4, "confidence": 0.88}
  },
  "usage": {"input_tokens": 312, "output_tokens": 0}
}

Pricing is $0.042 per million input tokens (output free), rate limit 250k tok/s (models page), so a routing decision costs roughly 1/50th–1/500th of an LLM-classifier call and returns in one round trip with the probabilities the routing engine actually needs. TypeSafe's own docs describe exactly this use: Intent routing and Confidence-gated routing.

People are already doing this on LiteLLM, which is where Bifrost users are being pulled:

• prismhq/jev-router — OpenAI-compatible LiteLLM proxy where clients send model: "jev-router" and a pre-call hook asks Jev which candidate from router.yaml should serve the request (capability filtering first, cheapest-eligible baseline when no TypeSafe key is set, fallback on failure).
• BerriAI/litellm#41607 — official /typesafe/{endpoint} passthrough with registry-priced spend tracking (jev-1.13.0, jev-latest, jev-preview), opened 2026-09-17.

Bifrost has the better substrate for this — native routing rules with scopes, chaining, budgets, and decision logging — but no way to put Jev in the decision seat. This is distinct from #7234, which asks for Jev as an inference provider (exposing /v1/systemone to callers). This issue is about using Jev inside the router. The two share the HTTP client and pricing entry; the provider alone does not give you routing.

Proposed solution

A Jev router: a third classifier for the routing engine, sitting next to the semantic and LLM classifiers in plugins/routing/complexity/, that publishes Jev's typed answers as CEL variables and lets a rule route on them.

1. Classifier. JevClassifier alongside SemanticClassifier / LLMClassifier: builds a minimised state from the request (same extraction the semantic classifier already does — last N user messages, plus cheap signals like has_tools, has_images, message_count, requested model), posts the configured questions to /v1/systemone, bounded by timeout (Jev's SDK default is 10 s; 1–2 s is realistic for routing). One HTTP call regardless of how many questions are asked — Jev evaluates them in parallel.

2. Config, in the same shape as the existing complexity_analyzer_config:

"complexity_analyzer_config": {
  "jev": {
    "enabled": true,
    "api_key": "env.TYPESAFE_API_KEY",
    "base_url": "https://api.typesafe.ai",
    "model": "jev-latest",
    "timeout": "2s",
    "message_history_count": 1,
    "count_toward_budgets": false,
    "min_confidence": 0.5,
    "questions": {
      "target":      {"type": "choice", "instructions": "Which model should serve this request",
                      "criteria": {"anthropic/claude-fable-5-1": "deep reasoning, long agentic coding",
                                   "openai/gpt-4o-mini": "short factual answers, rewrites",
                                   "gemini/gemini-2.5-flash": "summaries, translation, bulk text"}},
      "needs_tools": {"type": "noul",  "instructions": "The request requires function calling"},
      "complexity":  {"type": "score", "instructions": "How hard is this to answer well",
                      "criteria": ["trivial lookup", "some reasoning", "deep multi-step reasoning"]}
    }
  }
}

3. CEL surface. Every answer becomes a variable under a jev namespace; a rule references them like complexity_tier today, and classification runs only when a rule actually references jev.* (same laziness the docs promise for complexity_tier):

jev.target.choice == "openai/gpt-4o-mini" && jev.target.confidence > 0.6
jev.needs_tools > 0.5
jev.complexity.score >= 1.5 && team_name == "research"
jev.target.probabilities["anthropic/claude-fable-5-1"] > 0.3

Answers below min_confidence are unknown to the evaluator — the rule does not match and evaluation falls through to the next rule, exactly as complexity_tier behaves when the classifier abstains. That gives confidence-gated routing for free: a low-confidence decision drops to a default rule instead of being acted on.

4. Convenience target (the jev-router ergonomics from the LiteLLM project): a rule target {"provider": "$jev", "model": "$jev.target"} that resolves to whatever choice Jev returned, so a single rule covers a whole candidate pool without one rule per model. Weight/fallback semantics unchanged; if the choice is not a configured provider/model the rule is treated as non-matching.

5. Observability & spend. Record the raw answers (choice, probabilities, confidence, usage.input_tokens, resolved model id) in the routing decision log next to the matched phrase/similarity the semantic path already records, and price the call from a typesafe/jev-1.13.0 pricing entry at $0.042/Mtok input so count_toward_budgets works. Log-explorer and telemetry surfaces then show why a request went where it did, with a probability attached.

6. Optional: Jev as a Complexity-Router fallback. semantic.fallback: "jev" beside "llm", with a fixed three-level Score question mapped to SIMPLE | MEDIUM | COMPLEX. This is the smallest possible slice — it reuses the entire existing complexity_tier surface, and is where I'd start.

Alternatives considered

• Do it outside Bifrost (LiteLLM + jev-router, or an app-side pre-call to Jev). Works, but loses Bifrost's scopes, chaining, budgets and decision logs, and every client needs the extra hop.
• Use the existing LLM fallback classifier with a Jev-shaped prompt. Jev is not a chat model; /v1/chat/completions against it returns 400. Wrapping it in a chat shim would also throw away the probability distribution, which is the whole point.
• Only add Jev as an inference provider (#7234). Gives callers the /v1/systemone API through a virtual key, which is valuable, but the router still cannot consume the answers. A provider PR would be a natural prerequisite (shared client + pricing), so the two could be sequenced: provider first, router on top.
• Expand the reference-phrase classifier. More phrases still yields one of three labels and no calibrated confidence; it cannot answer "which of these five models" or "needs tools".

Area(s)

Plugins

Additional context

• Jev is in early access; TypeSafe rate limits are stated to be "adjusting dynamically" while they scale. The classifier must fail soft on 429/timeout (no tier published → fall through), and honour retry-after rather than retry inline on the request path.
• Privacy: routing sends a minimised request summary to a third party. The feature should be off by default, opt-in per config, and documented as such — jev-router calls this out explicitly and Bifrost should too. Hashing or truncating the state (roles + truncated text only, no system prompt, no tool results) keeps the payload small and limits exposure; the semantic classifier already never embeds system prompts or assistant replies, and the same rule should apply here.
• Aliases move: jev-latest → jev-1.13.0 today. Confidence thresholds tuned against one version may drift when the alias moves; the config should accept a pinned version, and the decision log should record the versioned model the response reports.
• References: Jev announcement · API quick start · Primitives: Choice / Score / Noul · Confidence · Models & pricing · prismhq/jev-router · BerriAI/litellm#41607 · Bifrost Complexity Router · Routing Rules
• Happy to contribute the fallback-classifier slice (item 6) and then the CEL jev.* surface if maintainers agree on the shape.