#4787·civitai

preview / smoke-tests: the generation cost quote fails on every PR based after ~2026-09-11 11:52

Author: ZacxDevCreated Sep 12, 2026Updated Sep 12, 2026

preview / smoke-tests is red on every PR based after ~2026-09-11 11:52. One spec fails — the generation cost quote — and it is a shared gate, so a permanently-red one trains everyone to click through it. That is what happened with #4171 (same class, different cause, now closed).

Found while merging an unrelated test-only PR (#4776) whose diff cannot reach this code path.

The failure

tests/preview-generation.spec.ts:193

Error: cost.total parsed from whatIfFromGraph response
  1 failed
  65 passed (1.8m)

Failed on the initial run and both retries, so it is not flaky-under-load.

It is NOT the timeout the spec's own NOTE warns about. That note says the test will time out if a future default ships with no model preselected. This is a different half: the whatIf response was observed and its body was captured — the spec throws a distinct, explicitly-worded error when capture fails, and that error did not appear. So the request fired and returned; extractCostTotal(body) simply found no cost.total in it. The response shape or content changed, rather than the request not happening.

Discriminating evidence — base-commit ancestry separates pass from fail cleanly

Same method as #4171.

PR merge-base with main base date preview / smoke-tests
#4777 ab24dc437c 2026-09-11 14:00 fail
#4776 fc39d4e5b3 2026-09-11 15:57 fail
#4771 a9713c6d80 2026-09-11 11:52 fail
#4739 6961693133 2026-09-08 18:50 pass
#4606 e92cf5fe4a 2026-09-03 16:40 pass
#4488 eedea41484 2026-08-28 23:31 pass

So the cause is on main, landing between 2026-09-08 18:50 and 2026-09-11 11:52 — not in any of the PRs.

#4776 is the cleanest control available: it changes exactly one file, src/components/Apps/AppActivityPage.browser.test.tsx. A browser test file is not shipped to the preview deployment and cannot alter runtime behaviour, let alone the generation cost path — yet it fails identically.

Candidates in that window (not verified — do not assume)

Commits on main touching the generation/orchestrator surface between those bases:

  • 0460b4c64b — run ZImage and Qwen on comfy, via @civitai/orchestration-client beta.104 ← a client bump is the strongest prior for a response-shape change
  • d6b48f5e49 — route the comfy-only ecosystems off sdcpp on the textToImage path
  • e680364460 — add Ideogram 4 generation support
  • 6bec49c2d8 — add ControlNet support for MiniMax H3
  • f0b1020fda — add GPT-Image-2.5 Flare and Sunburst

Several of these could change either the whatIf response shape or which workflow the default /generate form preselects — and the spec quotes whatever the default is.

Next probe

Two cheap reads, in order:

  1. Is this a test-shape break or a real pricing break? Capture the whatIf response body on a current preview and look at it directly — if cost.total moved or nested, the spec needs updating; if the response carries an error or a null quote, this is a product bug and the gate is doing its job.
  2. Bisect by base, not by code. Open a throwaway PR based on a commit between the two bounds and read the smoke verdict. #4739's base (pass) and #4771's base (fail) bracket it.

Do not conclude from an anonymous probe. #4171 recorded that trap for this suite: an unauthenticated request hits the auth gate and tells you nothing about the path under test.

Why this is worth its own issue

The spec asserts a numeric cost quote from the real pricing path. If the value is genuinely null rather than merely relocated, users are being quoted nothing on the default generate form — that is the failure mode the assertion exists to catch, and clicking through it because "smoke is always red" is exactly how it ships.

Closing condition

preview / smoke-tests reports pass on a PR whose merge-base is after the fix — mechanically checkable with gh pr checks <n>, no judgement required. Closed by whoever lands that fix. If instead the spec is found to be asserting a shape that legitimately changed, the closing condition is the same gate going green after the spec is updated, with a note here saying which of the two it was.

Checked by: gh pr checks <n> --repo civitai/civitai showing preview / smoke-tests = pass on a PR whose merge-base is after the fix — a command that exits 0, no judgement required. Equivalently, the merged PR that lands the fix.