feat(providers): add TypeSafe AI (Jev) as a provider + optional auto-routing classifier
Problem / Use Case
TypeSafe AI (https://typesafe.ai) ships "System One" models. Its flagship, Jev, does not generate text. You POST a state plus a map of typed questions (noul yes/no, choice, score) and get back typed answers with calibrated probabilities. Docs: https://docs.typesafe.ai/api
I want to call it through OmniRoute the same way I call every other provider: one key store, one base URL, usage and cost in the analytics tab, the same quota-aware fallback. Today I have to keep a separate key and a separate client for it.
There is a second, bigger win. OmniRoute's auto-routing intent classifier (open-sse/services/intentClassifier.ts) is keyword matching across nine languages. Jev is built for exactly that kind of decision. A single choice question over the prompt returns the intent plus a probability distribution and a confidence value, which OmniRoute could use to fall back to the keyword path when confidence is low.
Proposed Solution
Two parts. Part 1 stands on its own. Part 2 depends on it.
Part 1: typesafe provider (passthrough)
- New provider entry
typesafein the provider registry. API key auth, headerAuthorization: Bearer <key>, base URLhttps://api.typesafe.ai. - New route
POST /v1/systemonethat proxies the request body as-is tohttps://api.typesafe.ai/v1/systemoneusing the stored credential. Same pattern asPOST /v1/classify(Jina passthrough). GET /v1/modelsfrom TypeSafe lists the aliases (jev-latest,jev-preview). Expose them astypesafe/jev-latestetc. Versioned IDs likejev-1.13.0are accepted by the upstreammodelfield even when not listed, so don't reject unknownjev-*names.- Usage tracking: the response carries
usage.input_tokensandusage.output_tokens. Pricing is per input token only ($0.042 / Mtokfor Jev 1.13, output is free). The responsemodelfield reports the versioned ID that answered, so log that instead of the alias. - Rate limits: upstream returns
429and may sendretry-after. Feed that into the existing cooldown / fallback logic like any other provider. - Nothing chat-shaped. This provider should not appear in chat model lists or combos.
Part 2: Jev as the auto-routing classifier (opt-in)
- New option on the intent classifier config:
engine: "keywords" | "typesafe". Default stayskeywords. - When
typesafeis selected and atypesafecredential exists, send onechoicequestion with the six existingIntentTypevalues as criteria (code,math,reasoning,creative,simple,medium) and the prompt (plus system prompt) as state. - Use the returned confidence: below a configurable threshold, fall back to the keyword classifier. Record which engine made the call so
/v1/explain/routingcan show it. - If the TypeSafe call fails or times out, fall back to keywords. Routing must never block on it.
Alternatives Considered
- Custom OpenAI-compatible provider node: does not work, the request and response shape is not chat completions.
- Wrapping Jev in a chat-completions shim: loses the probability distribution, which is the whole point of the model.
- Keep using a separate client outside OmniRoute: works, but no shared key store, no usage tracking, no fallback.
Acceptance Criteria
POST /v1/systemonewith a storedtypesafekey returns the upstream response unchanged (status, body,usage).GET /v1/modelsincludestypesafe/jev-latestandtypesafe/jev-preview;typesafe/jev-1.13.0is passed through without a 404.- Analytics shows request count, input tokens, and cost for the
typesafeprovider. - Upstream
429triggers the normal provider cooldown and honorsretry-after. typesafemodels do not show up in chat completion model lists or combo builders.- With
engine: "typesafe"set, auto-routing calls Jev once per request, and/v1/explain/routingreports the engine, chosen intent, and confidence. - With the TypeSafe key missing, expired, or the request failing, routing falls back to keywords and the request still completes.
- Unit tests cover the passthrough handler and the classifier fallback path.
Area
Provider Support, Proxy / Routing, Analytics / Usage Tracking
Related Provider(s)
TypeSafe AI (Jev)
Additional Context
- API reference: https://docs.typesafe.ai/api
- Models, pricing, and rate limits: https://docs.typesafe.ai/models
- Intent routing pattern from their own docs: https://docs.typesafe.ai/patterns/intent-routing
- JS SDK if useful for types:
@typesafe-ai/sdk - TypeSafe notes their rate limits are adjusting dynamically right now, so the cooldown logic should trust
retry-afterover any hardcoded number.
Happy to test against my own key once there is a branch.
Source: diegosouzapw/OmniRoute