preview / smoke-tests: the generation cost quote fails on every PR based after ~2026-09-11 11:52
preview / smoke-tests is red on every PR based after ~2026-09-11 11:52. One spec fails — the generation cost quote — and it is a shared gate, so a permanently-red one trains everyone to click through it. That is what happened with #4171 (same class, different cause, now closed).
Found while merging an unrelated test-only PR (#4776) whose diff cannot reach this code path.
The failure
tests/preview-generation.spec.ts:193
Error: cost.total parsed from whatIfFromGraph response
1 failed
65 passed (1.8m)Failed on the initial run and both retries, so it is not flaky-under-load.
It is NOT the timeout the spec's own NOTE warns about. That note says the test will time out if a future default ships with no model preselected. This is a different half: the whatIf response was observed and its body was captured — the spec throws a distinct, explicitly-worded error when capture fails, and that error did not appear. So the request fired and returned; extractCostTotal(body) simply found no cost.total in it. The response shape or content changed, rather than the request not happening.
Discriminating evidence — base-commit ancestry separates pass from fail cleanly
Same method as #4171.
| PR | merge-base with main |
base date | preview / smoke-tests |
|---|---|---|---|
| #4777 | ab24dc437c |
2026-09-11 14:00 | fail |
| #4776 | fc39d4e5b3 |
2026-09-11 15:57 | fail |
| #4771 | a9713c6d80 |
2026-09-11 11:52 | fail |
| #4739 | 6961693133 |
2026-09-08 18:50 | pass |
| #4606 | e92cf5fe4a |
2026-09-03 16:40 | pass |
| #4488 | eedea41484 |
2026-08-28 23:31 | pass |
So the cause is on main, landing between 2026-09-08 18:50 and 2026-09-11 11:52 — not in any of the PRs.
#4776 is the cleanest control available: it changes exactly one file, src/components/Apps/AppActivityPage.browser.test.tsx. A browser test file is not shipped to the preview deployment and cannot alter runtime behaviour, let alone the generation cost path — yet it fails identically.
Candidates in that window (not verified — do not assume)
Commits on main touching the generation/orchestrator surface between those bases:
0460b4c64b— run ZImage and Qwen on comfy, via@civitai/orchestration-clientbeta.104 ← a client bump is the strongest prior for a response-shape changed6b48f5e49— route the comfy-only ecosystems off sdcpp on the textToImage pathe680364460— add Ideogram 4 generation support6bec49c2d8— add ControlNet support for MiniMax H3f0b1020fda— add GPT-Image-2.5 Flare and Sunburst
Several of these could change either the whatIf response shape or which workflow the default /generate form preselects — and the spec quotes whatever the default is.
Next probe
Two cheap reads, in order:
- Is this a test-shape break or a real pricing break? Capture the whatIf response body on a current preview and look at it directly — if
cost.totalmoved or nested, the spec needs updating; if the response carries an error or a null quote, this is a product bug and the gate is doing its job. - Bisect by base, not by code. Open a throwaway PR based on a commit between the two bounds and read the smoke verdict.
#4739's base (pass) and#4771's base (fail) bracket it.
Do not conclude from an anonymous probe. #4171 recorded that trap for this suite: an unauthenticated request hits the auth gate and tells you nothing about the path under test.
Why this is worth its own issue
The spec asserts a numeric cost quote from the real pricing path. If the value is genuinely null rather than merely relocated, users are being quoted nothing on the default generate form — that is the failure mode the assertion exists to catch, and clicking through it because "smoke is always red" is exactly how it ships.
Closing condition
preview / smoke-tests reports pass on a PR whose merge-base is after the fix — mechanically checkable with gh pr checks <n>, no judgement required. Closed by whoever lands that fix. If instead the spec is found to be asserting a shape that legitimately changed, the closing condition is the same gate going green after the spec is updated, with a note here saying which of the two it was.
Checked by: gh pr checks <n> --repo civitai/civitai showing preview / smoke-tests = pass on a PR whose merge-base is after the fix — a command that exits 0, no judgement required. Equivalently, the merged PR that lands the fix.
Source: civitai/civitai