#13987·OmniRoute

feat(providers): add TypeSafe AI (Jev) as a provider + optional auto-routing classifier

Author: cryptiklemurCreated Sep 17, 2026Updated Sep 17, 2026
Labelsenhancementproviders

Problem / Use Case

TypeSafe AI (https://typesafe.ai) ships "System One" models. Its flagship, Jev, does not generate text. You POST a state plus a map of typed questions (noul yes/no, choice, score) and get back typed answers with calibrated probabilities. Docs: https://docs.typesafe.ai/api

I want to call it through OmniRoute the same way I call every other provider: one key store, one base URL, usage and cost in the analytics tab, the same quota-aware fallback. Today I have to keep a separate key and a separate client for it.

There is a second, bigger win. OmniRoute's auto-routing intent classifier (open-sse/services/intentClassifier.ts) is keyword matching across nine languages. Jev is built for exactly that kind of decision. A single choice question over the prompt returns the intent plus a probability distribution and a confidence value, which OmniRoute could use to fall back to the keyword path when confidence is low.

Proposed Solution

Two parts. Part 1 stands on its own. Part 2 depends on it.

Part 1: typesafe provider (passthrough)

  • New provider entry typesafe in the provider registry. API key auth, header Authorization: Bearer <key>, base URL https://api.typesafe.ai.
  • New route POST /v1/systemone that proxies the request body as-is to https://api.typesafe.ai/v1/systemone using the stored credential. Same pattern as POST /v1/classify (Jina passthrough).
  • GET /v1/models from TypeSafe lists the aliases (jev-latest, jev-preview). Expose them as typesafe/jev-latest etc. Versioned IDs like jev-1.13.0 are accepted by the upstream model field even when not listed, so don't reject unknown jev-* names.
  • Usage tracking: the response carries usage.input_tokens and usage.output_tokens. Pricing is per input token only ($0.042 / Mtok for Jev 1.13, output is free). The response model field reports the versioned ID that answered, so log that instead of the alias.
  • Rate limits: upstream returns 429 and may send retry-after. Feed that into the existing cooldown / fallback logic like any other provider.
  • Nothing chat-shaped. This provider should not appear in chat model lists or combos.

Part 2: Jev as the auto-routing classifier (opt-in)

  • New option on the intent classifier config: engine: "keywords" | "typesafe". Default stays keywords.
  • When typesafe is selected and a typesafe credential exists, send one choice question with the six existing IntentType values as criteria (code, math, reasoning, creative, simple, medium) and the prompt (plus system prompt) as state.
  • Use the returned confidence: below a configurable threshold, fall back to the keyword classifier. Record which engine made the call so /v1/explain/routing can show it.
  • If the TypeSafe call fails or times out, fall back to keywords. Routing must never block on it.

Alternatives Considered

  • Custom OpenAI-compatible provider node: does not work, the request and response shape is not chat completions.
  • Wrapping Jev in a chat-completions shim: loses the probability distribution, which is the whole point of the model.
  • Keep using a separate client outside OmniRoute: works, but no shared key store, no usage tracking, no fallback.

Acceptance Criteria

  • POST /v1/systemone with a stored typesafe key returns the upstream response unchanged (status, body, usage).
  • GET /v1/models includes typesafe/jev-latest and typesafe/jev-preview; typesafe/jev-1.13.0 is passed through without a 404.
  • Analytics shows request count, input tokens, and cost for the typesafe provider.
  • Upstream 429 triggers the normal provider cooldown and honors retry-after.
  • typesafe models do not show up in chat completion model lists or combo builders.
  • With engine: "typesafe" set, auto-routing calls Jev once per request, and /v1/explain/routing reports the engine, chosen intent, and confidence.
  • With the TypeSafe key missing, expired, or the request failing, routing falls back to keywords and the request still completes.
  • Unit tests cover the passthrough handler and the classifier fallback path.

Area

Provider Support, Proxy / Routing, Analytics / Usage Tracking

Related Provider(s)

TypeSafe AI (Jev)

Additional Context

Happy to test against my own key once there is a branch.