There's an argument running about MCP right now.
You've probably seen it: a 400-point thread called "MCP is dead?" with real token numbers in it, four connected servers eating 21,077 tokens of context before anyone asks a question.
The argument is about what MCP costs.
Almost nobody in it has measured what agents actually do with the data a tool returns.
I ended up measuring that, not because I planned to, but because a maintainer refused to let me guess.
The question nobody wanted to answer with opinions I contribute to CNCF Jaeger's MCP server.
Last April I proposed exposing service performance metrics (latencies, call rates, error rates) as an MCP tool, and hit an immediate design fork: what shape should the output be?
Option one, summary rows: pre-aggregated stats per service, compact, cheap.
Option two, per-bucket time series: the raw points, roughly 720 of them per service at default resolution, expensive but complete.
I asked which the maintainer preferred.
The answer, verbatim, from the issue thread: This type of decision should not be based on opinion, but on benchmarks with a real agent troubleshooting some issues and using this MCP tool to access metrics, where you could do A/B testing of different output formats.
Fair.
So I built the A/B.
The setup Everything below is public in jaeger-mcp-bench, including the harness, the tasks, the scorer, and a research log of everything that went wrong.
The fixture is Jaeger v2 with the spanmetrics connector, hotrod generating traffic, and Prometheus behind it, snapshotted so every run sees identical metric state.
In front of the metrics API sits a thin bench server with exactly one switch: .
No new semantics, just the shape of what comes back.
Six troubleshooting tasks, and this part matters: three were chosen because I predicted summary would win them (point questions: current latency, ranking, threshold checks) and three because I predicted series would win (temporal questions: spike detection, correlation, trend).
Designing the tasks to favor both arms keeps the deck unstacked against your own hypothesis.
Two agents, and not stripped-down tool loops: Claude Sonnet through the Claude Code CLI and Gemini 2.5 Pro through the gemini CLI, real system prompts and all, because that's what production agents actually are.
Three trials per cell, 72 trials total, cells run in randomized order (seed 42) so fixture drift can't correlate with either arm.
Scoring is programmatic against ground truth, not vibes.
What 72 trials said arm correct wrong declined claude / series 18 0 0 gemini / series 17 0 1 claude / summary 10 1 7 gemini / summary 11 0 7 The result I expected was wrong answers.
Bad format, wrong conclusions, embarrassing agent behavior, good story.
That is not what happens.
Across 72 trials there was exactly one wrong commitment, and it traced to a bug in my own bench server, not to the format (disclosed in RESULTS.md; it biased against summary, and I'd rather report my own bug than a fake finding).
The real difference lives in the decline rate.
Given summary rows, agents said some version of "I cannot determine this from the available data" seven times more often than with series, almost entirely on the temporal questions.
And they were right to decline: aggregation had destroyed the time axis the question needed.
You cannot localize a spike in an average.
Agents don't get it wrong when you underfeed them.
They give up, politely and correctly.
The stats hold up under correction: Claude's summary-versus-series gap is significant after Bonferroni (p=0.001 against a 0.0125 alpha).
Gemini's, at p=0.016, sits right on the boundary, and I'd rather say that than round it in my favor.
Point queries were a wash, exactly as predicted: when the question needs one number, format doesn't matter.
So series won.
But that's not the useful lesson The tool shipped with per-bucket series as the decision, backed by data instead of my taste.
Fine.
The useful lesson is about what "expensive output"