AG-UI MessagesSnapshot: convert_messages_snapshot drops tool call name and arguments (ToolCall(name='unknown_tool', args={}))
- I have checked the documentation and related resources and couldn't resolve my bug.
Describe the bug
convert_messages_snapshot() (via AGUIEventCollector._handle_messages_snapshot, src/ragas/integrations/ag_ui.py:636) rebuilds tool calls from a MessagesSnapshotEvent with getattr(tc_obj, "name", "unknown_tool") and getattr(tc_obj, "args", {}). In ag_ui.core, ToolCall has no top-level name / args; both are nested under ToolCall.function (FunctionCall.name: str, FunctionCall.arguments: str — a JSON string). Both getattr calls therefore always fall back, so every tool call recovered from a snapshot comes back as ToolCall(name='unknown_tool', args={}) and the real tool name and arguments are silently discarded.
Root cause: src/ragas/integrations/ag_ui.py:636-637 at commit 298b68274234c060deacab3cf5fb52aa3a20e885 reads the flat shape where the AG-UI object is nested. The sibling writer in the same file, convert_messages_to_ag_ui (lines 925-931), builds the nested form AGUIToolCall(id=..., function=FunctionCall(name=tc.name, arguments=json.dumps(tc.args))), so a Ragas → AG-UI → Ragas round-trip loses every tool name and argument. The streaming path in the same class (_handle_tool_call_end) does parse function.arguments-style JSON, so only the snapshot path is affected.
Ragas version: v0.4.3-8-g298b682 (editable pip install -e . of commit 298b68274234c060deacab3cf5fb52aa3a20e885)
Python version: 3.12.14
ag-ui-protocol: 0.1.9 (pyproject.toml pins ag-ui = ["ag-ui-protocol>=0.1.9"]; the nesting was verified in 0.1.9, 0.1.21 and 0.1.22)
OS: macOS 26.6.2 (Darwin arm64)
Code to Reproduce
# real `pip install -e .` of ragas @ 298b682, no shim
from ragas.integrations.ag_ui import convert_messages_snapshot, extract_tool_calls
from ag_ui.core import MessagesSnapshotEvent, AssistantMessage, ToolCall as AGToolCall, FunctionCall
snap = MessagesSnapshotEvent(messages=[
AssistantMessage(id="a1", content="Checking", tool_calls=[
AGToolCall(id="tc1", function=FunctionCall(name="get_weather", arguments='{"city":"SF"}'))]),
])
msgs = convert_messages_snapshot(snap)
print(msgs)
print("extract:", extract_tool_calls(msgs))The same handler runs for any MESSAGES_SNAPSHOT event routed through convert_to_ragas_messages(), which run_ag_ui_row() uses (verified with a RunStartedEvent + MessagesSnapshotEvent stream).
Error trace
Actual output of the script above:
[AIMessage(content='Checking', metadata=None, type='ai', tool_calls=[ToolCall(name='unknown_tool', args={})])]
extract: [ToolCall(name='unknown_tool', args={})]A second, independent run against a two-message snapshot (assistant message + tool result) reproduced the same loss:
AIMessage 'Let me check the weather.' [ToolCall(name='unknown_tool', args={})]
ToolMessage 'Sunny, 72F' None
extracted: [ToolCall(name='unknown_tool', args={})]
A) convert_to_ragas_messages -> [('AIMessage', [ToolCall(name='unknown_tool', args={})])]
B) writer produced FunctionCall -> name='get_weather' arguments='{"city": "SF"}'
B) round-trip tool_calls -> [[ToolCall(name='unknown_tool', args={})]]Expected behavior
convert_messages_snapshot() should return ToolCall(name='get_weather', args={'city': 'SF'}) for the snapshot above, and the round-trip through convert_messages_to_ag_ui() should preserve the tool name and arguments.
Basis:
ag-ui-protocol(checked in 0.1.9 — the minimum allowed by this repo — plus 0.1.21 and 0.1.22) declaresclass ToolCall(ConfiguredBaseModel): id: str; type: Literal["function"] = "function"; function: FunctionCallandclass FunctionCall(ConfiguredBaseModel): name: str; arguments: str. Neither class has a top-levelnameorargs; constructing a flatToolCall(id=..., name=..., args=...)raisesValidationError: function / Field required.- The surrounding code already agrees on the nested layout:
convert_messages_to_ag_ui()writesFunctionCall(name=tc.name, arguments=json.dumps(tc.args))in the same file, and_handle_tool_call_end()parsesfunction.argumentsJSON for the streaming path. - Running the same fallback logic against a real object of the pinned minimum version (
ag-ui-protocol==0.1.9):
ToolCall fields: ['id', 'type', 'function']
FunctionCall fields: ['name', 'arguments']
nested: get_weather '{"city": "SF"}'
ragas-style getattr -> unknown_tool {}Additional context
- Related issues/PRs: none found. Searched this repo's issues and PRs (open + closed) for
ag_ui,unknown_toolandmessages_snapshot— no matches, and no open PR currently touchessrc/ragas/integrations/ag_ui.py. - I'm happy to be assigned, and can open a PR that reads the nested
functionshape (name plusjson.loadsof the arguments string, with a fallback when it is not valid JSON).
Source: vibrantlabsai/ragas