[BUG] Claude Code tracing records no token usage or cost for assistant messages that call a tool or only think

[!WARNING] Before submitting a PR, please make sure that:

• A maintainer has triaged this issue and applied the ready label
• This issue has no assignee
• No duplicate PR exists

PRs not meeting these requirements may be automatically closed.

Summary

In libs/typescript/integrations/claude-code/src/tracing.ts, createLlmAndToolSpans creates the llm span only inside:

// tracing.ts, line 378
if (textContent.trim() && !toolUses.length) {

setTokenUsageAttribute(llmSpan, usage) and calculateCost(model, usage) both live inside that branch and nowhere else, so an assistant message that does not pass the condition contributes no token usage and no cost at all. Two common cases fail it:

1. Any message that calls a tool. !toolUses.length excludes it.
2. Any thinking-only message. extractContentAndTools in _internal.ts accumulates textContent only from parts of type: 'text', so a thinking block leaves textContent empty.

The live path in the same package already handles both. liveTracing.ts line 156 gates on:

if (textContent.trim() || hasThinking(content)) {

and emits its tool spans from a separate loop outside that branch, so a thinking-only or tool-calling message still gets its llm span there. Only the transcript path used by the Stop hook has the restriction, so the two paths disagree about the same conversation.

Because mlflow.trace.cost is the sum of the span costs in the trace, the trace-level cost is short by the same proportion as the usage.

Measured on one real Claude Code session (@mlflow/claude-code 0.4.0, 283 distinct assistant message.id values across 681 transcript entries). Of 681 assistant entries, 83 pass the condition, 376 are skipped for calling a tool, and 222 are skipped for containing only thinking. Claude Code writes one transcript entry per content block and stamps the same usage on each entry of a message, so the honest unit is the message: 58 of 283 messages (20.5%) emit at least one text-only entry, and the rest record nothing.

Summing each message's usage once:

Roughly four fifths of the cache reads on a heavily agentic session are never recorded, which is where almost all of the cost sits.

Expected behavior

Every assistant message that carries a usage payload should produce one llm span with its token usage and its cost, whether it wrote prose, only produced a thinking block, or went straight to a tool call. That is what liveTracing.ts already does, and it would make the two paths agree.

Tool spans are created in a separate if (toolUses.length) block a few lines below, so relaxing this condition does not change or duplicate them.

One caveat for whoever fixes this

Simply dropping the condition would over-count, because the unit is wrong. Claude Code writes several transcript entries per assistant message that share one message.id, and in the sampled session 227 of 228 multi-entry messages carried byte-identical usage on every entry. Emitting one llm span per entry therefore records the same usage several times: simulated against the same session, that yields 140,264,488 cache-read tokens against an actual 58,917,275, or 238% of the truth.

A correct fix needs to group transcript entries by message.id and emit one llm span per message, rather than one per entry.

How to reproduce

1. Install the mlflow-tracing Claude Code plugin (@mlflow/claude-code 0.4.0) and point it at an MLflow experiment.
2. Run a Claude Code session that does real work, so most turns are extended thinking followed by tool calls rather than plain prose replies.
3. Open the resulting traces. Count the llm spans against the assistant messages in the session transcript at ~/.claude/projects/<project>/<session-id>.jsonl, filtering to type == "assistant" and grouping by message.id.
4. Only the messages whose content includes a text block with no tool_use block have an llm span. Those messages are the only contributors to mlflow.chat.tokenUsage and mlflow.trace.cost, and on an agentic session they are a small minority.

Related

#25826 makes exactly this change for the sibling Codex integration: "Emit an LLM span for every model response, including tool-only responses that contain no assistant text." The Claude Code integration has the same gap in tracing.ts, plus the thinking-only case above.

What component(s) does this affect?

• area/tracing: MLflow Tracing features, tracing APIs, and LLM tracing functionality