Re-enable the snapshot acceptance test block once Pipes snapshot creates stop timing out
The snapshot block has been removed from the acceptance-test matrix in
.github/workflows/11-test-acceptance.yaml because publishing a snapshot to
turbot-ops/clitesting times out. This issue tracks putting it back.
Why it was disabled
tests/acceptance/test_files/snapshot.bats publishes three snapshots to
turbot-ops/clitesting (one per output format). Creates against that workspace
take longer than the API gateway's 60s limit, so the publish returns
504 Gateway Timeout and the test fails.
Measured 2026-08-17, three consecutive POST requests to
/api/v0/org/turbot-ops/workspace/clitesting/snapshot with a 23.6KB payload:
| attempt | status | time |
|---|---|---|
| 1 | 504 | 60.458s |
| 2 | 504 | 60.444s |
| 3 | 504 | 60.443s |
All three are the gateway's ceiling, not a measurement of how long the create takes — the real duration is only known to be above 60s.
Reads, deletes and workspace lookups against the same workspace all answer in under a second. Only creates are slow.
Why this leaks snapshots
The server completes the insert even when the gateway has already given up. Two
of the three probe requests above landed as available rows in the workspace
while the client received a 504 and no snapshot ID. Nothing client-side can
clean those up, because the ID is what a delete needs and it was never returned.
That is a second, independent leak from the one where a failing assertion skips
an inline delete. Moving the delete into teardown() does not help here — the
teardown has no ID either.
What the disabled tests actually covered
The three tests assert CSV, JSON and table output formatting in snapshot mode.
As the file's own comment at tests/acceptance/test_files/snapshot.bats:14
says, they are "not snapshot creation/upload" tests — the upload is incidental
to exercising --snapshot. So the coverage lost is output formatting, not the
publish path.
What needs to happen before re-enabling
- Snapshot creates against the CI workspace need to return under 60s. The workspace holds ~2,018 visible snapshots. An earlier theory that create latency scales with row count now looks doubtful: a large number of rows were deleted a week before the measurements above and the latency did not move.
- A scheduled cleanup for orphaned snapshots, since a timed-out create leaves a row no test can identify.
Once creates are reliably fast, restore the - "snapshot" entry to the
test_block matrix.
Source: turbot/steampipe