LangChain CallbackHandler implements on_llm_error/on_tool_error/on_retriever_error but not on_chain_error, so an errored chain span is never exited and its trace is never posted

❗BEFORE YOU BEGIN❗ Are you on discord? We'd love to have you asking questions on discord instead: https://discord.com/invite/a3K9c8GRGt

Describe the bug

deepeval/integrations/langchain/callback.py implements on_llm_error, on_tool_error and on_retriever_error, but not on_chain_error. LangChain dispatches on_chain_error(error, run_id=..., parent_run_id=...) from the except BaseException as e branch of Runnable.invoke, so an errored chain only reaches the inherited protocol no-op. The chain span created in on_chain_start is therefore never passed to exit_current_context: it stays in trace_manager.active_spans with status=TraceSpanStatus.SUCCESS and error=None, and because a root chain span has parent_uuid=None, the code path that would call trace_manager.end_trace() never runs. In bare (no @observe) usage end_trace() is what calls post_trace(), so the entire trace is silently never shipped to Confident AI. Leaked nested-chain spans also keep other_active_spans non-empty, so an enclosing trace never ends either.

To Reproduce

Environment:

• deepeval commit 1e4f9e6e9f0bdf01a88f9e4d69d0eb5d0de634c1 (main)
• langchain_core 1.6.3
• Python 3.12
• macOS 15 (Darwin 25.6.0)

Steps to reproduce the behavior:

1. Attach deepeval's CallbackHandler() to any LangChain/LangGraph runnable.
2. Start a chain (on_chain_start), then have the chain raise — LangChain calls run_manager.on_chain_error(e) and re-raises.
3. Inspect trace_manager.

Minimal reproduction (langchain_core is stubbed below only so the module imports; the code under test is the real tree code):

from uuid import uuid4
from deepeval.integrations.langchain.callback import CallbackHandler
from deepeval.tracing import trace_manager

handler = CallbackHandler()
print("handler defines on_chain_error :", "on_chain_error" in CallbackHandler.__dict__)

rid = uuid4()
handler.on_chain_start({"name": "RunnableSequence"}, {"question": "hi"}, run_id=rid)
span = trace_manager.get_span_by_uuid(str(rid))
trace_uuid = span.trace_uuid

# LangChain dispatches this on a chain failure; deepeval never overrides it,
# so the base-class no-op is what actually runs.
CallbackHandler.on_chain_error(
    handler, ValueError("pinecone exploded"), run_id=rid, parent_run_id=None
)

span = trace_manager.get_span_by_uuid(str(rid))
print("span still present             :", span is not None)
print("span.status                    :", span.status)
print("span.error                     :", span.error)
print("trace still open               :", trace_manager.get_trace_by_uuid(trace_uuid) is not None)

Actual output

Verifier 1 (langchain_core 1.6.3):

on_chain_error defined on handler: False
hooks dispatched: ['on_chain_start', 'on_chain_error']
active_spans count : 1
  span 01a0ab0d status=TraceSpanStatus.SUCCESS error=None parent=None
post_trace calls   : 0
-- success control (same handler, no raise): post_trace calls : 1
-- repo's own test, unpatched: WARNING langchain_core.callbacks.manager:manager.py:443 Error in RecordingCallbackHandler.on_chain_error callback: AssertionError("assert AgentSpan(uuid='01a0ab0f-...', status=<TraceSpanStatus.SUCCESS...>, error=None, parent_uuid=None ...) is None")  [test still reports PASSED]

Verifier 2:

declared in CallbackHandler.__dict__: {'on_chain_start': True, 'on_chain_end': True, 'on_chain_error': False, 'on_llm_error': True, 'on_tool_error': True, 'on_retriever_error': True}
resolved on_chain_error owner: ChainManagerMixin.on_chain_error
chain raised: pinecone exploded
leaked span name='RunnableSequence' class=AgentSpan status=TraceSpanStatus.SUCCESS error=None end_time=None
leaked span name='boom' class=BaseSpan status=TraceSpanStatus.SUCCESS error=None end_time=None
active_traces (still open): ['10f06e73-d282-45b2-8cfe-fdda7cd636f9']   post_trace called: []
[success path] leaked spans: []   open traces : []   post_trace  : ['2a65a12b-...']

Expected behavior

on_chain_error should mirror the sibling error handlers already implemented in the same class: mark the span TraceSpanStatus.ERRORED, set span.error, and call exit_current_context, so the span is removed and the trace is posted.

Concrete basis:

• langchain_core dispatches it: libs/core/langchain_core/runnables/base.py:2311 — except BaseException as e: run_manager.on_chain_error(e); raise in _call_with_config (mirrored in RunnableSequence.invoke; dispatch sites also at 2354/2411/2595/3617/4192). Protocol definition at libs/core/langchain_core/callbacks/base.py:189. ChainManagerMixin.on_chain_error's body is a docstring only (a verified no-op).
• Sibling branches in the same file do it correctly: on_llm_error (callback.py:714), on_tool_error (callback.py:848) and on_retriever_error (callback.py:927) each set TraceSpanStatus.ERRORED + span.error and then call exit_current_context. Observer.__exit__ does the same (deepeval/tracing/tracing.py:1266-1267).
• deepeval/tracing/tracing.py:340-365 shows end_trace() is what calls post_trace().
• deepeval's own test asserts the intended contract: tests/test_integrations/test_langgraph/test_create_task.py:97-108 subclasses CallbackHandler and, after super().on_chain_error(...), asserts trace_manager.get_span_by_uuid(rid) is None when parent_run_id is None, under the docstring "…a chain_error event is recorded, and the chain span is removed." That assertion fails at runtime today.

Desktop:

• OS: macOS 15 (Darwin 25.6.0)
• Version: deepeval main @ 1e4f9e6, langchain_core 1.6.3, Python 3.12

Additional context

Root cause: deepeval/integrations/langchain/callback.py:355 (on_chain_end) has no on_chain_error counterpart in the class, so LangChain's chain-error dispatch falls through to ChainManagerMixin.on_chain_error's no-op and the root chain span is never exited — and exit_current_context is the only path that calls trace_manager.end_trace() for parent_uuid=None root spans, which is what triggers post_trace().

Reachability: any LangChain or LangGraph app that attaches deepeval's CallbackHandler() and raises inside a chain — e.g. an agent executor whose tool raises, a RunnableSequence step raising, or any chain-level exception.

Collision checks found no existing issue or PR for this (gh search issues/prs for on_chain_error and chain error callback in this repo both returned empty).

Happy to open a PR adding on_chain_error(self, error, *, run_id, parent_run_id=None, **kwargs) mirroring on_tool_error (look up the span by str(run_id), return if absent, then inside self._ctx(run_id, parent_run_id) set the span status/error and call exit_current_context), if that approach works for you.