#40518·langchain

UsageMetadataCallbackHandler silently drops usage_metadata when model_name is missing

Author: ammar-iitmCreated Sep 16, 2026Updated Sep 17, 2026
Labelsbugcorestandard-testsexternal

Submission checklist

  • This is a bug, not a usage question.
  • I added a clear and descriptive title that summarizes this issue.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangChain rather than my code.
  • The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
  • This is not related to the langchain-community package.
  • I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Package (Required)

  • langchain
  • langchain-openai
  • langchain-anthropic
  • langchain-classic
  • langchain-core
  • langchain-model-profiles
  • langchain-tests
  • langchain-text-splitters
  • langchain-chroma
  • langchain-deepseek
  • langchain-exa
  • langchain-fireworks
  • langchain-groq
  • langchain-huggingface
  • langchain-mistralai
  • langchain-nomic
  • langchain-ollama
  • langchain-openrouter
  • langchain-perplexity
  • langchain-qdrant
  • langchain-xai
  • Other / not sure / general

Related Issues / PRs

PR #40513 (fix, ready — references this issue once it's reopened)

Reproduction Steps / Example Code (Python)

from langchain_core.callbacks import get_usage_metadata_callback
from langchain_core.language_models import GenericFakeChatModel
from langchain_core.messages import AIMessage
from langchain_core.messages.ai import UsageMetadata

usage = UsageMetadata(input_tokens=1, output_tokens=2, total_tokens=3)
llm = GenericFakeChatModel(messages=iter([AIMessage("hi", usage_metadata=usage)]))

with get_usage_metadata_callback() as cb:
    llm.invoke("hello")
    print(cb.usage_metadata)  # {} -- usage silently lost

Error Message and Stack Trace (if applicable)

Description

UsageMetadataCallbackHandler.on_llm_end (in langchain_core/callbacks/usage.py) only records token usage when the response's response_metadata also contains a model_name key:

if usage_metadata and model_name: ...

Any chat model whose AIMessage carries real usage_metadata but doesn't set response_metadata["model_name"] (custom BaseChatModel subclasses, or some third-party integrations) has its usage silently dropped — callback.usage_metadata stays {} with no error or warning. Since the entire purpose of this callback/context manager is usage tracking, this defeats it in a way that's easy to miss.

Expected behavior: Usage should still be tracked when model_name is absent, e.g. bucketed under an "unknown" key, rather than dropped entirely.

Proposed fix: Change the condition to only require usage_metadata, and key the dict with model_name or "unknown". This is backward compatible: models that do set model_name (the common case) are unaffected.

I have a fix with a regression test ready — PR #40513.

System Info

System Information

OS: Darwin OS Version: Darwin Kernel Version 27.0.0 Python Version: 3.14.6 (main, Jun 10 2026, 10:03:53) [Clang 21.0.0]

Package Information

langchain_core: 1.6.1 langsmith: 0.11.1 langchain_protocol: 0.0.18 langchain_tests: 1.1.9 langchain_text_splitters: 1.1.2

Social handles (optional)

No response