Feature: use TypeSafe Jev as the complexity decision engine for complexity routing
Feature Description
Add an optional TypeSafe Jev integration as the decision engine behind complexity routing (complexity_routing_enabled). When a Jev API key is configured, one typed /v1/systemone call per routable request decides the tier: a score for how much reasoning the task needs, a choice over the configured tiers, and a noul flag for deep-reasoning work, each with a confidence value. Code then applies a confidence gate (e.g. low confidence or a deep-reasoning flag escalates one tier) instead of relying solely on the current keyword/structural scorer.
Motivation
The current scorer misclassifies real agent traffic. Two examples from existing issues:
- #2575: Claude Code injects
<system-reminder>blocks into the user turn, so the scorer classifies harness instructions instead of the user's request and nearly everything lands on the most expensivereasoningtier. - #1620: auto mode fails to switch models by task difficulty at all.
A typed decision model reads the request semantically rather than matching imperative/constraint register, so injected instruction preambles stop looking like hard tasks. Confidence scores also give a principled second axis: act on the tier when confident, escalate when not.
Proposed Solution
- New optional setting (e.g.
TYPESAFE_API_KEYenv / gateway config). Unset key = current behavior, zero regression for existing users. - Pre-routing phase sends a minimized request summary (recent messages, truncated; harness-injected blocks stripped, cf. #2575) to
/v1/systemonewith three typed questions: complexityscore, tierchoice, deep-reasoningnoul. - Confidence gate in code:
choiceconfidence below threshold bumps one tier up;noulabove threshold bumps one tier up. - Fail-open: any Jev failure (timeout, HTTP error) falls back to the existing scorer/baseline so routing never blocks a request.
- Decision metadata (tier, confidence, escalated) logged per request for observability.
Alternatives Considered
- External sidecar proxy: I currently run a small Jev relay in front of the gateway that rewrites the model per request. It works, but it pins an explicit model name upstream, which bypasses the dashboard routing table and
fallback_routes; keeping it in sync requires polling the Manifest DB. Native support removes that entire workaround class. - Keep improving the heuristic scorer: cheaper (no third-party call), but heuristics cannot distinguish user intent from harness-injected instruction text, which is the root cause of #2575.
Source: mnfst/manifest