Surface pricing-feed degradation that silently weakens the routing budget
Author: adilhafeezCreated Jul 21, 2026Updated Jul 21, 2026
Labelsenhancementarea/routing
Follow-up from #982.
The routing budget is only as good as the pricing feed, and two degradation modes are currently invisible to operators:
- Empty rates map. If the cost feed (models.dev / DO catalog) is unreachable at startup or refresh, the rates map is empty and every switch fails open (
reason="no_pricing"). Startup is bounded by client timeouts (added in #982), but the only signal is a fetch log line — there is no metric to alert on. - Cached-rate fallback. When a feed doesn't publish a cached-read rate for a model, Plano silently estimates
cached_rate = input_rate x cache_read_discount(default 0.1). If the provider's real discount differs, switch costs and the never-switch baseline are mispriced with no indication. (The DO catalog now publishescache_read_input_price_per_million, so this mainly affects models.dev entries and DO models missing the field.)
Proposed:
- A gauge for cost-feed health (models loaded, last successful fetch timestamp) suitable for alerting.
- A counter or per-decision span attribute marking when the switch-cost calculation used the fallback discount rather than a published cached rate.
- Docs: call out both modes in the routing_budget config reference.
Related idea worth considering here: a static/local pricing override (JSON/YAML file merged over the feed) to remove the hard runtime dependency on an external pricing service — useful for self-hosted models whose IDs no public feed knows.
Source: katanemo/plano