BUG: pricing resolver returns the aggregator route for equally priced models (test failing on main)
tests/test_pricing.py::test_resolves_common_bare_model_names fails on current main (v1.6.2, 52b1923):
assert resolve_litellm_model("grok-4.5") == "xai/grok-4.5"
E AssertionError: assert 'openrouter/x-ai/grok-4.5' == 'xai/grok-4.5'
Full run: 1740 passed, 1 failed.
Cause
LiteLLM 1.90.1 (the version pinned in uv.lock) lists grok-4.5 under three routes with identical pricing:
key in litellm.model_cost |
litellm_provider |
input | output |
|---|---|---|---|
xai/grok-4.5 |
xai |
2e-06 | 6e-06 |
openrouter/x-ai/grok-4.5 |
openrouter |
2e-06 | 6e-06 |
perplexity/xai/grok-4.5 |
perplexity |
2e-06 | 6e-06 |
In strix/report/pricing.py, resolve_litellm_model collects the matching keys, and when the prices tie it returns matches[0] from an alphabetically sorted list:
matches = sorted(key for key in model_cost if key.endswith(f"/{name}"))
...
if len(matches) == 1 or len(prices) == 1:
return matches[0]
openrouter/... sorts before xai/..., so the aggregator route now wins over the provider's own. Nothing in the repo changed — the LiteLLM registry gained the aggregator entries.
Suggested fix
Among equally priced matches, prefer the canonical route: the one shaped exactly {litellm_provider}/{name}. An explicitly aggregator-qualified name such as x-ai/grok-4.5 should still resolve to the aggregator, since that is what the caller asked for.
I have this implemented and tested locally (1741 passed, ruff/mypy/pyright clean) and will open a PR referencing this issue.
Environment
- Strix
main@52b1923(v1.6.2) - Python 3.12, litellm 1.90.1 (from
uv.lock) - Linux
Source: usestrix/strix