[BUG] bmad-build-auto step-04: the follow-up-review flag cannot return false, and one review layer's quota manufactures findings
Description
Two coupled properties of bmad-build-auto's step-04 make the review loop non-convergent in practice. Measured over one epic driven to completion: 9 stories, 21 triage passes, 733 findings recorded.
1. followup_review_recommended cannot return false.
step-04 → Finalize computes it as: true if any patched finding was high severity, or if 3 × medium + 1 × low >= 5, counting only findings triaged patch.
Passes in this epic patched 8–24 findings each. That threshold is met every time. 21 of 21 passes with patch > 0 returned true. The single false in the whole epic came from a story whose passes patched nothing at all — every finding routed to bad_spec.
Worth flagging because bmad-loop's own source comment states the opposite expectation:
pre-#2580, every review pass patched findings and recommended another pass; the upstream severity-scored flag has since made that the exception.
On this codebase the scored flag is not the exception. It fires on every pass that does any work.
2. The blind-hunter default layer is given a quota, not a question.
From the shipped customize.toml:
Find at least ten issues to fix or improve. […] If you have zero findings, re-check and keep thinking; do not stop with an empty list.
21 passes × a floor of 10 ⇒ at least 210 findings exist because the prompt requires them, independent of what the diff contains.
Combined outcome. Of the 733 findings, 271 were rejected at P3/P4 and 125 deferred at P3/P4 — 396, more than half, went nowhere. The rejection rate climbs monotonically with each pass:
| Triage pass | Findings | Rejected |
|---|---|---|
| 1 | 164 | 20% |
| 2 | 220 | 41% |
| 3 | 207 | 56% |
| 4 | 133 | 64% |
Severity decays with it: pass 1 carried 1 P0 and 11 P1; pass 4 carried 1 P1 against 106 P3.
Steps to reproduce
- BMAD-METHOD 6.11.0,
bmad-build-auto, default review layers. - Drive a multi-story epic through repeated invocations. (We orchestrate with bmad-loop 0.11.0 at
review.trigger = "always",limits.max_followup_reviews = 1, but neither property depends on the orchestrator — both are computed inside the skill.) - After each pass, read
followup_review_recommendedin the spec frontmatter and the counts in## Review Triage Log.
Expected behavior
followup_review_recommended separates a pass whose findings warrant an independent second opinion from one whose findings do not. A review layer reports what it found.
Actual behavior
The flag returns true whenever the pass patched anything at all, so it carries no signal. blind-hunter returns at least ten findings whether or not ten exist.
Suggested direction
Offered as direction, not as a prescribed fix.
- Score the flag on findings left unresolved rather than on the count of patches applied. A pass that fixed everything it found is evidence of convergence, not evidence that another pass is needed.
- Make
blind-hunter's ten a ceiling rather than a floor, or drop the count entirely. - Related seam: step-01 already routes a
status: donespec to step-04 as a fresh independent review, so the skill can tell a first pass from a re-review. The layer set is identical in both. A per-layer gate on that distinction would let a project run cheap layers inline and reserve expensive ones for the independent pass.
What was not measured
Per-layer attribution of the 733 findings. Our run logs are terminal redraw captures and collapse subagent output, so the ≥210 floor is derived from the prompt's own instruction times the pass count, not from counting blind-hunter's output directly.
Environment
BMAD-METHOD 6.11.0 (bmm module), macOS, Claude Code adapter, orchestrated by bmad-loop 0.11.0.
Source: bmad-code-org/BMAD-METHOD