Output quality is important when working with AI coding agents, but true efficiency comes from getting work done quickly, efficiently, and with the right context.
That’s why token count of individual interactions alone isn’t a meaningful measure of efficiency.
The goal shouldn’t be to use fewer tokens, but to tap into the right amount of context to move a task forward.
A concise tool response can sometimes require additional calls or work if it leaves out information the agent needs, ultimately making the task slower and more expensive.
That’s why we want to optimize for the outcome rather than the tool call.
This post examines four changes in GitHub Copilot that put that principle into practice: Preserve useful context while reducing repetitive output.
Remove formatting that adds no value to the task.
Shorten instructions without changing useful behavior.
Deliver completed background work without an extra retrieval step.
Possible changes were evaluated offline using agentic coding benchmarks.
The most promising changes were then validated through controlled online experiments before shipping.
The examples in this post come from GitHub Copilot CLI.
Multiple other Copilot products, such as the GitHub Copilot app and Copilot code review, use the same underlying harness and also become more efficient through these improvements.
Figure 1: Four independent A/B experiments using the same AI-credit metric.
The segments are shown together for comparison; their effects are not necessarily strictly additive.
The local metric trap It’s common to shorten the output from each tool call as a way to reduce agent costs.
RTK (Rust Token Killer) is a utility that shortens shell output before an agent reads it.
We evaluated its effect on GitHub Copilot using our agentic coding benchmarks.
In our harness and benchmark configuration, RTK shortened some responses, but when the omitted text mattered, the model sometimes reopened the original output or reran the command to recover what it needed.
Those recovery steps added turns and carried more context forward.
The individual tool response was shorter, but on average, the task used more tokens and took longer.
We saved tokens locally and spent more globally.
Figure 2: A shorter tool response can make the completed task more expensive when missing details force the agent to reread output, rerun commands, and carry more context forward.
This result applies to the integration and workloads we tested, not to every RTK configuration or to output compression in general.
This meant that tokens per tool call is the wrong objective.
An efficiency change has to be evaluated across the complete task, from the user’s request through the final result.
More useful was to look at what can we remove without making the model repeat work.
Compress noise, preserve useful information The goal was to shorten repetitive output while preserving the context an agent needs to complete its task without retracing steps.
Analysis of benchmark runs showed that install, build, test, and lint output often contains repetitive noise, while source-like output and arbitrary command results are more likely to contain the information an agent needs.
That analysis informed a selective output compressor, informed in part by RTK and similar approaches.
The prototype was evaluated on agentic coding benchmarks and a range of open source repositories, exercising their build, test, and lint systems.
Early versions were too aggressive.
They made the model repeat work or read the full saved output, increasing end-to-end cost and reducing task success.
For example, we initially compressed but removed that filter after benchmark tasks showed agents reopening the original output to recover missing information.
Those early failures led to a three-part policy: Preserve source-like and arbitrary output.
Commands such as , , , and arbitrary scripts are returned unchanged.
Reorganize search results without dropping c