UsageMetadataCallbackHandler silently drops usage_metadata when model_name is missing
Submission checklist
- This is a bug, not a usage question.
- I added a clear and descriptive title that summarizes this issue.
- I used the GitHub search to find a similar question and didn't find it.
- I am sure that this is a bug in LangChain rather than my code.
- The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
- This is not related to the langchain-community package.
- I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.
Package (Required)
- langchain
- langchain-openai
- langchain-anthropic
- langchain-classic
- langchain-core
- langchain-model-profiles
- langchain-tests
- langchain-text-splitters
- langchain-chroma
- langchain-deepseek
- langchain-exa
- langchain-fireworks
- langchain-groq
- langchain-huggingface
- langchain-mistralai
- langchain-nomic
- langchain-ollama
- langchain-openrouter
- langchain-perplexity
- langchain-qdrant
- langchain-xai
- Other / not sure / general
Related Issues / PRs
PR #40513 (fix, ready — references this issue once it's reopened)
Reproduction Steps / Example Code (Python)
from langchain_core.callbacks import get_usage_metadata_callback
from langchain_core.language_models import GenericFakeChatModel
from langchain_core.messages import AIMessage
from langchain_core.messages.ai import UsageMetadata
usage = UsageMetadata(input_tokens=1, output_tokens=2, total_tokens=3)
llm = GenericFakeChatModel(messages=iter([AIMessage("hi", usage_metadata=usage)]))
with get_usage_metadata_callback() as cb:
llm.invoke("hello")
print(cb.usage_metadata) # {} -- usage silently lost
Error Message and Stack Trace (if applicable)
Description
UsageMetadataCallbackHandler.on_llm_end (in langchain_core/callbacks/usage.py) only records token usage when the response's response_metadata also contains a model_name key:
if usage_metadata and model_name: ...
Any chat model whose AIMessage carries real usage_metadata but doesn't set response_metadata["model_name"] (custom BaseChatModel subclasses, or some third-party integrations) has its usage silently dropped — callback.usage_metadata stays {} with no error or warning. Since the entire purpose of this callback/context manager is usage tracking, this defeats it in a way that's easy to miss.
Expected behavior: Usage should still be tracked when model_name is absent, e.g. bucketed under an "unknown" key, rather than dropped entirely.
Proposed fix: Change the condition to only require usage_metadata, and key the dict with model_name or "unknown". This is backward compatible: models that do set model_name (the common case) are unaffected.
I have a fix with a regression test ready — PR #40513.
System Info
System Information
OS: Darwin OS Version: Darwin Kernel Version 27.0.0 Python Version: 3.14.6 (main, Jun 10 2026, 10:03:53) [Clang 21.0.0]
Package Information
langchain_core: 1.6.1 langsmith: 0.11.1 langchain_protocol: 0.0.18 langchain_tests: 1.1.9 langchain_text_splitters: 1.1.2
Social handles (optional)
No response
Source: langchain-ai/langchain