`RequestUsage.extract` swallows all exceptions and returns a fabricated zero usage

This issue was researched and posted by Claude Code (claude-opus-5) on behalf of Jun-Dir Liew (@jliew).

Review level: the mechanism and the reproducer below were verified by a Claude Code session against pydantic-ai==2.35.0 with genai-prices==0.1.4 in a clean venv — the reproducer output is pasted verbatim from that run. pydantic_ai/usage.py and genai_prices/{types,units}.py were read directly. The 1.107.5 incident and the genai-prices bisect are from our own runs. Jun-Dir directed the work and approved filing.

Initial Checks

• I'm using the latest version of Pydantic AI — the incident was on 1.107.5; the mechanism and reproducer below are verified on 2.35.0
• I've searched for my issue in the issue tracker before opening this issue

Description

RequestUsage.extract wraps its whole provider loop in a bare except Exception: pass, then falls through to return cls(details=details) — a zeroed RequestUsage. pydantic_ai/usage.py, current main / v2.35.0:

for provider_id, provider_api_url in [(None, provider_url), (provider, None), (provider_fallback, None)]:
    try:
        provider_obj = get_snapshot().find_provider(None, provider_id, provider_api_url)
        _model_ref, extracted_usage = provider_obj.extract_usage(data, api_flavor=api_flavor)
        return cls(**{k: v for k, v in extracted_usage.__dict__.items() if v is not None}, details=details)
    except Exception:
        pass
return cls(details=details)

The facts:

• On any exception, extract returns RequestUsage() — all zeros.
• RequestUsage() is also what a genuine zero-token call produces (free tier, fully cached response). Callers cannot tell the two apart.
• Nothing is raised, warned, or logged.
• The caller cannot opt out of this behaviour, and cannot recover the swallowed exception.

For contrast, in this same file cost uses the opposite convention: it is None when unavailable — so "unknown" stays distinguishable from a genuine zero — and UsageLimits._warn_if_cost_unavailable emits CostNotFoundWarning when that blocks a cost_limit. Usage extraction failing is both indistinguishable and silent.

The try covers three statements with three different error contracts

Only the first is a question the loop is entitled to answer by moving on:

What this costs when it fires

We hit this on pydantic-ai==1.107.5. genai-prices 0.1.0 added output_reasoning_tokens to the reported key set; 1.x's RequestUsage was a @dataclass(kw_only=True) with a generated __init__, so the splat raised TypeError — swallowed — zeros.

Two Gemini text steps recorded 0 → 0 tokens and $0.00, having each recorded a normal few cents the day before the upgrade. No exception, no warning, and $0.00 is a plausible reading when part of your traffic is free-tier. Every Anthropic step in the same runs was correct, because Anthropic folds thinking into output_tokens and never sets the attribute — so only models reporting reasoning tokens separately were affected, and the failure was invisible in aggregate. Bisected: genai-prices 0.0.62–0.0.73 extract 27191/14864, every 0.1.x returns 0/0.

That specific trigger is already fixed here — #6683 (v2.17.0) made RequestUsage accept arbitrary fields, and main now floors genai-prices>=0.1.0. This report is about the except, which is unchanged.

Why the swallow outlasts each instance

genai-prices' Usage is a dynamic container whose valid keys come from the pricing-data unit registry, so the shape crossing this boundary can widen without a code change in either library. Recent instances where usage silently went missing:

• #6683 — output_reasoning_tokens vs. the fixed dataclass (the one above).
• #7681's findings table — Z.AI output_reasoning_tokens "went silently missing" after a provider entry changed which extractor matched; and Bedrock Mantle RequestUsage.cost.

Each was fixed as an instance. The except Exception: pass is what makes the next one silent too. (#7681 also added latest-versions-canary.yml, which is what surfaced two of these — but it tests dependency resolution, not this code path.)

Consumers of the fabricated value

UsageLimits.check_tokens (token limits silently never trip), cost_limit, OTel gen_ai.usage.* attributes, Logfire cost dashboards, and any billing reconciliation built on result.usage.

Suggested fix

The ask is don't be silent, not always raise — usage extraction being best-effort is reasonable. Options, in increasing strictness:

1. warnings.warn on any non-LookupError, and keep returning zeros. This alone would have surfaced our incident on day one.
2. Additionally propagate when it is cls(**...) that raised, i.e. when pydantic-ai rejected data genai-prices produced — that is a bug rather than a best-effort miss.
3. A strict/lenient switch.

Not asking for output_reasoning_tokens to be declared as a field — #7094 covers that.

Happy to write the regression test and validate whatever fix you land. It needs no dependency pinning — the reproducer below is the test. tests/test_usage_limits.py and tests/models/test_xai.py are the test files that currently touch RequestUsage.extract. Per CONTRIBUTING I won't open a PR unless this gets assigned.

Minimal, Reproducible Example

Against pydantic-ai==2.35.0 (which pulls genai-prices==0.1.4). No API key, no network. The provider lookup matches on the first candidate in both cases — these are not provider misses.

import warnings
from types import SimpleNamespace

import pydantic_ai.usage as usage_mod
from pydantic_ai.usage import RequestUsage


class FakeProvider:
    def __init__(self, outcome):
        self.outcome = outcome

    def extract_usage(self, data, *, api_flavor='default'):
        if self.outcome == 'raise':
            raise RuntimeError('provider extractor raised')
        return 'some-model', self.outcome


class FakeSnapshot:
    def __init__(self, outcome):
        self.outcome = outcome

    def find_provider(self, model_ref, provider_id, provider_api_url):
        return FakeProvider(self.outcome)  # a real match, on the first candidate


def show(label, outcome):
    usage_mod.get_snapshot = lambda: FakeSnapshot(outcome)
    with warnings.catch_warnings(record=True) as caught:
        warnings.simplefilter('always')
        result = RequestUsage.extract(
            {}, provider='google', provider_url='https://example.invalid', provider_fallback='google'
        )
    print(f'{label}\n  returned    : {result!r}\n  has_values(): {result.has_values()}\n'
          f'  warnings    : {[str(w.message) for w in caught]}\n')


show('extract_usage() raises:', 'raise')
show('extracted object carries `requests`:',
     SimpleNamespace(input_tokens=27191, output_tokens=14864, requests=1))
print(f'a genuinely free call:\n  returned    : {RequestUsage()!r}\n')

try:
    RequestUsage(input_tokens=27191, output_tokens=14864, requests=1)
except Exception as exc:
    print(f'the swallowed exception in case 2: {type(exc).__name__}: {exc}')

Output:

extract_usage() raises:
  returned    : RequestUsage()
  has_values(): False
  warnings    : []

extracted object carries `requests`:
  returned    : RequestUsage()
  has_values(): False
  warnings    : []

a genuinely free call:
  returned    : RequestUsage()

the swallowed exception in case 2: AttributeError: property 'requests' of 'RequestUsage' object has no setter

Case 2 is not arbitrary. RequestUsage.requests is a read-only @property and UsageBase.__init__ assigns every kwarg with setattr, so any extracted object carrying requests zeroes the usage silently. requests is a unit in the pricing data; the two libraries agree today only because genai_prices.units filters exactly that one key out of the reported set:

self._reported_usage_keys_in_order = tuple(usage_key for usage_key in units if usage_key != 'requests')

On genai-prices==0.1.4 that registry reports 58 usage keys; RequestUsage declares 7. The rest arrive as untyped setattr attributes (which is what #7094 is about).

To be clear: requests is not the subject of this issue. It is used here only because it is a convenient trigger that still fires on current main. A one-line change in either library would close that particular door, and the except would go on converting the next mismatch into a silent zero in exactly the same way. Case 1 of the reproducer, where extract_usage simply raises, needs no key mismatch at all — that is the behaviour this issue is about.

Python, Pydantic AI & LLM client version

• Python: 3.11.8
• Pydantic AI: incident observed on 1.107.5; mechanism and reproducer verified on 2.35.0
• LLM provider SDK: google-genai (Gemini, via a Cloudflare AI Gateway base URL); genai-prices==0.1.4