We run ~100 LLM agents unattended on local models.
Last week we found one document that had been rewritten 245 times in 5 days — every attempt rejected.
A sibling document: 225 times.
Combined, about 470 wasted generations, all burned on the same two files.
Here is the autopsy, with the actual numbers.
The loop Our pipeline is simple: a producer agent writes a document, a reviewer agent checks it against a contract (minimum length, required sections, no placeholder junk), and rejected work goes back with fix instructions.
The rejected document was a key-management (KMS) implementation spec — 4,452 characters, perfectly on-topic.
The reviewer's verdict: "The request was a 3-line email triage response (LOCK / VERDICT / REASON), but the answer is a long KMS spec.
Rewrite as 3 lines only." One problem.
We grepped the document: the words "LOCK", "VERDICT", and the name of the triage service appear zero times in it.
The reviewer had invented the request.
Why the loop never ended Two contracts collided: The reviewer's fix instruction: output 3 lines only The producer's output contract: minimum 600 characters No output can satisfy both.
So the producer failed the contract, got re-queued, produced again, failed again — 245 times.
Our retry cap counted reviews, but a contract-failed output never reaches review.
The give-up mechanism existed; it just watched the wrong counter.
Root cause: the reviewer never saw the request Our review prompt contained the artifact body (first 4,000 chars) and the output format.
It never contained the original request.
We asked a model "does this match the request?" without telling it what the request was.
A model asked to judge against information it doesn't have will hallucinate that information.
Ours did, confidently, 245 times' worth.
Bonus failure: we truncated long documents to 4,000 characters before review without saying so, and reviewers marked them "thin — cut off mid-sentence." The cut was ours, not the producer's.
How common was it?
We audited all 2,038 reviews on file for concrete terms (product names, format tokens) that appear in the review but nowhere in the reviewed document.
Result: 4 contaminated reviews — 0.2%.
That's the uncomfortable lesson: a 0.2% hallucination rate produced 470 wasted runs, because nothing ever gave up.
Low rate × infinite retries = unbounded damage.
The rate is not the risk; the loop is.
The fixes (all mechanical) Pass the original request into the review prompt.
If it can't be extracted, the prompt now says: "do NOT guess the request — say it is unknown and judge the artifact on its own." Declare truncation. "First 4,000 of 8,784 chars — the cut is ours." Reject impossible instructions at the review's own exit gate.
A review demanding "N lines only" while the production contract requires 600+ chars now fails as a review and never enters the queue.
Count consecutive contract failures, not just reviews, and park the item for a human after 5 — with the last verdict and fix instruction attached, so the human can see why in one glance.
Each fix ships with a test we deliberately broke to confirm it fails.
If you run agents unattended The checker that catches broken outputs in this story (empty text, language leakage, placeholder junk, contract violations) is free on npm: honto-contract — it passed 600 downloads last week, so somebody besides us finds this useful now.
The unattended-operation checklist and three of our watchdog templates are free (email-gated): Unattended-Operation Kit The full production set (7 templates — cron registry, silent-zero watch, heartbeat, output contracts: the exact ones in this story) is available on the same page. (https://gxcafe.co.jp/harness-kit/?utm_source=devto&utm_medium=article&utm_campaign=templates-pro).
Honest note: we have no customers yet.
Everything above is exactly what we run on ourselves, measured on our own failures.