Gemini 3.8 Flash via OpenRouter: agents finish clean while producing nothing — turns end with no tool call and coverage silently collapses
Summary
With openrouter/google/gemini-3.8-flash, child agents complete successfully while
producing nothing — a measurable fraction of their turns end with no tool call at
all, so they never call record_coverage or agent_finish. The run reports
scan_completed: true and exit_reason: finished_by_tool; the only trace is
agent_recorded_no_coverage in the coverage record.
This is adjacent to #1295 but a distinct and quieter failure mode. There, agents
die loudly with ModelRefusalError from a content filter and are marked failed. Here,
on a healthy provider, agents are never marked failed at all — they simply return
empty turns and the scan finishes normally with a near-empty ledger.
Measurement, with matched controls
Same target, same instruction file, same -m deep --scope-mode full --max-turns 120,
same strix build, same host, runs minutes apart. Only STRIX_LLM changed.
Counting the "ended a turn without a lifecycle tool call" log line:
| Model | Turns with no tool call | Requests | Rate |
|---|---|---|---|
openrouter/z-ai/glm-5.3 |
0 | 897 | 0.0 % |
claude-cli/claude-opus-4-8 |
0 | 404 | 0.0 % |
openrouter/google/gemini-3.8-flash (run 1) |
18 | 707 | 2.5 % |
openrouter/google/gemini-3.8-flash (run 2) |
12 | 244 | 4.9 % |
Zero for both non-Gemini models. The rate roughly doubled on the second Gemini run, which was the one with a healthy provider.
Effect on the scan record
| Gemini run 1 | Gemini run 2 | |
|---|---|---|
completeness.complete |
false | false |
Agents raising agent_recorded_no_coverage |
2 | 4 of 4 |
summary.surfaces_reviewed |
19 | 6 |
outcomes.ruled_out |
5 | 0 |
| Findings filed | 0 | 0 |
scan_completed / exit_reason |
true / finished_by_tool |
true / finished_by_tool |
| Provider transient errors | 30 | 3 |
| Agents marked failed | 1 | 0 |
In run 2 every child agent recorded nothing, so the six ledger entries were all
written by the root. They are umbrella claims spanning whole route trees rather than
specific probes, and ruled_out is zero — nothing was actually cleared. The root then
called stop_agent on a still-working child
(reason='Stopping agent to reconcile assessment results') and finish_scan.
For scale, the comparison models on the identical target recorded 50 and 68 surfaces
with 15 and 25 ruled_out respectively.
Why run 2 is the interesting one
Run 1 had an obvious excuse: 30 transient provider errors and 15 OpenRouter 500s
(OpenrouterException - Internal Server Error, Metadata: {'error_type': 'unmapped'}),
one arriving mid-stream where litellm cannot fail over, killing an agent after all
5 retries. The two non-Gemini models went through the same OpenRouter endpoint in the
same session with zero provider errors.
Run 2 had a healthy provider — 3 transient errors, all recovered on retry, zero agent failures — and was worse in every coverage dimension. So the provider instability is not the cause; the empty turns are, and they are what survives when the provider is fine.
Ruled out
- Content filtering (#1295's mechanism). Zero occurrences of
ModelRefusalError,content filter,refused to produce,Response withheld, or any safety/blocked signature in either run'sstrix.log. - Tool-count / schema rejection. All four runs passed an identical 41 tools
per agent. No
INVALID_ARGUMENT, noBadRequest, no API 400 in either Gemini run.
Possible relation to #719
#719 describes litellm converting a provider response.failed into a normal, empty,
successful-looking turn — "no exception, no retry, and no log signal." That is exactly
the observable shape here. #719 is filed against the Responses API bridge with Azure,
so this may be a different code path reaching the same outcome; noting it in case the
fix generalises.
External reports describe the same behaviour for Gemini Flash in tool-using loops: empty assistant responses after tool-call completion, where tools execute but the model never continues the turn.
Suggested handling
The scan-level problem is not that a model returns an empty turn — it is that a run
which lost every child agent's output still reports scan_completed: true with no
error. Options, roughly increasing in strength:
- Count and surface empty turns. They are already logged; promote the count into
run.json/coverage so a reader can see "N turns produced no tool call" instead of inferring it from an absent ledger. - Treat a sustained empty-turn rate as a failure condition. The forced tool continuation already fires; if an agent exceeds a threshold, mark it degraded rather than letting it finish clean.
- Make
agent_recorded_no_coverageblockcomplete: truemore loudly — arguably an agent that produced nothing should make the run's status, not only its coverage caveats, reflect that.
Environment: strix 1.7.0, Python 3.12, Linux container, local-code target (a multi-tenant web app), non-interactive mode, OpenRouter provider.
Happy to supply the anonymised coverage records or log excerpts for any of the four runs.
Source: usestrix/strix