Integrate Fugusashi-based intelligent model routing (Like Sakana Fugu. But Free.)

I would like to request to add support for using Fugusashi v1.3 as an intelligent model router in freellmapi. It is inspired by Sakana Fugu breakthrough

freellmapi already supports multiple upstream providers, models, retries, and failure handling. Fugusashi could complement this by analyzing each incoming request and selecting the most appropriate configured model or model group based on the prompt.

For example:

• Coding requests could be routed to a coding-optimized model.
• Creative-writing requests could be routed to a creative model.
• Simple questions could be routed to a lower-cost model.
• Complex reasoning requests could be routed to a more capable model.
• Requests could fall back to the existing weighted or health-based routing when the router is unavailable.
• Fugusashi is an open-source ModernBERT-based model-routing engine designed for prompt classification and model selection. It supports CPU inference and provides model-selection decisions with confidence information.

Desired Behavior Add an optional intelligent-routing mode to a freellmapi group:

routing:  strategy: intelligent  router:    type: fugusashi    endpoint: http://fugusashi:8000    timeout_ms: 100    minimum_confidence: 0.65    fallback_strategy: weighted

The router would receive relevant request information, such as:

{  "model": "auto",  "messages": [    {      "role": "user",      "content": "Write a Python function that parses a CSV file."    }  ]}

It would return a routing decision such as:

{  "model": "coding-model",  "confidence": 0.91,  "reason": "The request requires code generation."}

freellmapi would then route the request to the matching configured model, provider, or subgroup.

Suggested requirements

1. Add intelligent as an optional routing strategy.
2. Support Fugusashi through:
   • An HTTP endpoint, and/or
     • A locally hosted Python sidecar/container.
3. Map Fugusashi output classes to freellmapi models or groups.
4. Support confidence thresholds.
5. Fall back automatically to the existing routing strategy when:
   • The router times out.
     • The router is unavailable.
     • The confidence score is below the configured threshold.
     • No matching model or group exists.
6. Preserve existing health checks, retries, cooldowns, rate limits, and credential scheduling.
7. Add optional logging showing:
   • Selected model or group.
     • Router confidence.
     • Routing latency.
     • Fallback reason, if applicable.
8. Provide an option to disable request-content logging because prompts may contain sensitive information.
9. Allow users to override automatic routing by explicitly specifying a model.
10. Support streaming, tool calls, multimodal requests, and other request types without breaking current behavior.

Example configuration

routing:  strategy: intelligent
  intelligent:    provider: fugusashi    endpoint: http://fugusashi:8000/route    timeout: 100ms    minimum_confidence: 0.65    fallback: weighted
    mappings:      coding: coding-model      creative: creative-model      reasoning: reasoning-model      simple: economical-model

Alternatively, the configuration could map router classes to freellmapi groups:

mappings:  coding: group-coding  creative: group-creative  reasoning: group-reasoning  factual_qna: group-general

Acceptance criteria

• Existing freellmapi routing behavior remains unchanged when intelligent routing is disabled.
• A request using model: auto can be routed to a configured model or group based on its content.
• Explicit model selections bypass the intelligent router.
• Router failures do not make freellmapi unavailable.
• Low-confidence decisions use the configured fallback strategy.
• Routing decisions and fallback events are observable in the request logs or metrics.
• The feature works with both local Fugusashi deployments and remote Fugusashi-compatible endpoints.
• The implementation does not require sending prompts to an external service unless the administrator explicitly configures one.

Motivation

This would allow freellmapi to make routing decisions based not only on traffic weight, account health, or availability, but also on the type and complexity of each request. It could reduce costs, improve latency, and make better use of specialized models while retaining freellmapi’s existing failover and load-balancing features.