[BUG] bmad-build-auto step-04: the follow-up-review flag cannot return false, and one review layer's quota manufactures findings

Author: tronghieuCreated Aug 24, 2026Updated Sep 10, 2026

Description

Two coupled properties of bmad-build-auto's step-04 make the review loop non-convergent in practice. Measured over one epic driven to completion: 9 stories, 21 triage passes, 733 findings recorded.

1. followup_review_recommended cannot return false.

step-04 → Finalize computes it as: true if any patched finding was high severity, or if 3 × medium + 1 × low >= 5, counting only findings triaged patch.

Passes in this epic patched 8–24 findings each. That threshold is met every time. 21 of 21 passes with patch > 0 returned true. The single false in the whole epic came from a story whose passes patched nothing at all — every finding routed to bad_spec.

Worth flagging because bmad-loop's own source comment states the opposite expectation:

pre-#2580, every review pass patched findings and recommended another pass; the upstream severity-scored flag has since made that the exception.

On this codebase the scored flag is not the exception. It fires on every pass that does any work.

2. The blind-hunter default layer is given a quota, not a question.

From the shipped customize.toml:

Find at least ten issues to fix or improve. […] If you have zero findings, re-check and keep thinking; do not stop with an empty list.

21 passes × a floor of 10 ⇒ at least 210 findings exist because the prompt requires them, independent of what the diff contains.

Combined outcome. Of the 733 findings, 271 were rejected at P3/P4 and 125 deferred at P3/P4 — 396, more than half, went nowhere. The rejection rate climbs monotonically with each pass:

Triage pass Findings Rejected
1 164 20%
2 220 41%
3 207 56%
4 133 64%

Severity decays with it: pass 1 carried 1 P0 and 11 P1; pass 4 carried 1 P1 against 106 P3.

Steps to reproduce

  1. BMAD-METHOD 6.11.0, bmad-build-auto, default review layers.
  2. Drive a multi-story epic through repeated invocations. (We orchestrate with bmad-loop 0.11.0 at review.trigger = "always", limits.max_followup_reviews = 1, but neither property depends on the orchestrator — both are computed inside the skill.)
  3. After each pass, read followup_review_recommended in the spec frontmatter and the counts in ## Review Triage Log.

Expected behavior

followup_review_recommended separates a pass whose findings warrant an independent second opinion from one whose findings do not. A review layer reports what it found.

Actual behavior

The flag returns true whenever the pass patched anything at all, so it carries no signal. blind-hunter returns at least ten findings whether or not ten exist.

Suggested direction

Offered as direction, not as a prescribed fix.

  • Score the flag on findings left unresolved rather than on the count of patches applied. A pass that fixed everything it found is evidence of convergence, not evidence that another pass is needed.
  • Make blind-hunter's ten a ceiling rather than a floor, or drop the count entirely.
  • Related seam: step-01 already routes a status: done spec to step-04 as a fresh independent review, so the skill can tell a first pass from a re-review. The layer set is identical in both. A per-layer gate on that distinction would let a project run cheap layers inline and reserve expensive ones for the independent pass.

What was not measured

Per-layer attribution of the 733 findings. Our run logs are terminal redraw captures and collapse subagent output, so the ≥210 floor is derived from the prompt's own instruction times the pass count, not from counting blind-hunter's output directly.

Environment

BMAD-METHOD 6.11.0 (bmm module), macOS, Claude Code adapter, orchestrated by bmad-loop 0.11.0.

Source: bmad-code-org/BMAD-METHOD