core: `AsyncBaseTracer` drops `name` in `on_tool_start` and `response` in `on_llm_error`, diverging from `BaseTracer`
Submission checklist
- This is a bug, not a usage question.
- I added a clear and descriptive title that summarizes this issue.
- I used the GitHub search to find a similar question and didn't find it.
- I am sure that this is a bug in LangChain rather than my code.
- The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
- This is not related to the langchain-community package.
- I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.
Package (Required)
- langchain
- langchain-openai
- langchain-anthropic
- langchain-classic
- langchain-core
- langchain-model-profiles
- langchain-tests
- langchain-text-splitters
- langchain-chroma
- langchain-deepseek
- langchain-exa
- langchain-fireworks
- langchain-groq
- langchain-huggingface
- langchain-mistralai
- langchain-nomic
- langchain-ollama
- langchain-openrouter
- langchain-perplexity
- langchain-qdrant
- langchain-xai
- Other / not sure / general
Related Issues / PRs
#22151 (introduced with_alisteners / AsyncRootListenersTracer)
Reproduction Steps / Example Code (Python)
import asyncio
from langchain_core.language_models import GenericFakeChatModel
from langchain_core.messages import AIMessageChunk
from langchain_core.outputs import ChatGenerationChunk
from langchain_core.tools import tool
from langchain_core.tracers.base import AsyncBaseTracer, BaseTracer
# --- 1. on_tool_start ignores `name` -----------------------------------------
@tool
def add(a: int, b: int) -> int:
"""Add two numbers."""
return a + b
# --- 2. on_llm_error ignores `response` --------------------------------------
class SyncTracer(BaseTracer):
def __init__(self) -> None:
super().__init__()
self.error_outputs = []
def _persist_run(self, run) -> None:
pass
def _on_llm_error(self, run) -> None:
self.error_outputs.append(run.outputs)
class AsyncTracer(AsyncBaseTracer):
def __init__(self) -> None:
super().__init__()
self.error_outputs = []
async def _persist_run(self, run) -> None:
pass
async def _on_llm_error(self, run) -> None:
self.error_outputs.append(run.outputs)
class FailsMidStream(GenericFakeChatModel):
def _stream(self, *args, **kwargs):
yield ChatGenerationChunk(message=AIMessageChunk(content="partial"))
raise ValueError("boom")
async def main() -> None:
names = []
add.with_listeners(on_end=lambda run: names.append(run.name)).invoke(
{"a": 1, "b": 2}, config={"run_name": "renamed"}
)
async def aon_end(run) -> None:
names.append(run.name)
await add.with_alisteners(on_end=aon_end).ainvoke(
{"a": 1, "b": 2}, config={"run_name": "renamed"}
)
print("run names (sync, async):", names)
# Actual: ['renamed', 'add']
# Expected: ['renamed', 'renamed']
sync_tracer, async_tracer = SyncTracer(), AsyncTracer()
model = FailsMidStream(messages=iter([]))
try:
list(model.stream("hi", config={"callbacks": [sync_tracer]}))
except ValueError:
pass
try:
async for _ in model.astream("hi", config={"callbacks": [async_tracer]}):
pass
except ValueError:
pass
print("sync error outputs has generations:", "generations" in sync_tracer.error_outputs[0])
print("async error outputs:", async_tracer.error_outputs[0])
# Actual: True / {}
# Expected: True / {'generations': [...partial output...], ...}
asyncio.run(main())Error Message and Stack Trace (if applicable)
Description
AsyncBaseTracer in langchain_core/tracers/base.py has two handlers that don't forward arguments their sync counterparts in BaseTracer do. Any async tracer is affected, including AsyncRootListenersTracer, which backs Runnable.with_alisteners.
AsyncBaseTracer.on_tool_startignoresname. It acceptsnameas an explicit parameter but never passesname=nameto_create_tool_run, and sincenameis a named parameter it isn't in**kwargseither. The tool run's name falls back to the serialized tool name (or"Unnamed"), so arun_nameset via config is lost on the async path.BaseTracer.on_tool_startdoes passname=name, as do the other asyncon_*_starthandlers.AsyncBaseTracer.on_llm_errordropsresponse. When a stream fails partway,BaseChatModel/BaseLLMcallrun_manager.on_llm_error(error, response=LLMResult(...))with the partial generations.BaseTracer.on_llm_errorforwardsresponse=kwargs.pop("response", None)to_errored_llm_run, which stores those generations inrun.outputs. The async version calls_errored_llm_run(error=error, run_id=run_id)withoutresponse, so async tracers recordrun.outputs == {}and lose the partial output.
Proposed fix: pass name=name to _create_tool_run in AsyncBaseTracer.on_tool_start, and response=kwargs.pop("response", None) to _errored_llm_run in AsyncBaseTracer.on_llm_error, plus unit tests asserting sync/async parity for both. I'd be glad to open a PR with the fix and tests if a maintainer assigns this to me.
System Info
- langchain-core 1.6.3 (also current
master) - Python 3.10.16
- macOS 26.6 (arm64)
Social handles (optional)
No response
Source: langchain-ai/langchain