E2E: cron-interactive nightly flakes on 30 s cron-fire timeouts, and continue-on-error swallows the red
Author: chiga0Created Sep 3, 2026Updated Sep 17, 2026
Labelspriority/P2type/bugscope/testingscope/ci-cd
Summary
Two of the three cases in integration-tests/interactive/cron-interactive.test.ts intermittently time out at 30 s waiting for a cron fire/notification. The job itself sets continue-on-error: true, so a red does not even fail the run — it surfaces only as a job-list entry. Combined with the fact that the job runs only on schedule/workflow_dispatch, nothing in the normal path ever sees it.
Evidence
- Red runs (same signature in both): https://github.com/QwenLM/qwen-code/actions/runs/33356793651 (nightly, 2026-08-31) and https://github.com/QwenLM/qwen-code/actions/runs/33702961416 (dispatch, 2026-09-03). Totals:
Test Files 1 failed (1),Tests 2 failed | 1 passed (3). - Signatures:
Error: Timeout (30000ms) waiting for: first cron fire "Cron: CRONTICK99"— caseuser input takes priority over cronError: Timeout (30000ms) waiting for: cron notification "Cron: PONG7742"— caseloop fires inline in conversation
- Green on the same job: nightly 33469433430 (09-01), 33590493339 (09-02), 33292207570 (08-30). So: 1 failure in the last 4 nightlies, 1 in 5 dispatched runs.
- The failure's screen dump shows the prompt that was sent (
Call cron_create with expression "*/1 * * * *" and prompt "CRONTICK99" and recurring true), i.e. the expected once-a-minute fire did not arrive inside the 30 s budget.
Why this stays invisible
.github/workflows/e2e.yml:377-378gates theisolated-nightlymatrix toscheduleorworkflow_dispatch; onpushtomainit never runs..github/workflows/e2e.yml:379setscontinue-on-error: true, so a failing leg is reported as success at the run level.
Two mechanisms worth separating first
- A test-budget bug. Cron granularity is one minute, the wait is 30 s, so whether the first fire lands in budget depends on where the run started relative to the minute boundary. That predicts a roughly position-dependent flake rate rather than a random one, and it predicts the first fire is the one that times out (consistent with the
CRONTICK99signature). - A behavior gap.
cron_createwas never called, or the recurring job never fired at all. The second signature (cron notification "Cron: PONG7742") needs checking against its own screen dump before being attributed to either mechanism.
I have not reproduced this locally — the interactive legs need model credentials, which this environment lacks. The distinction above is a reading of the failure output, not a verified diagnosis.
Acceptance
- Whichever mechanism it is: the cases become deterministic (drive the clock past a boundary, or size the wait to cover it) rather than re-rolled.
- The
continue-on-errorchoice is reconsidered for this matrix — if a nightly leg is meant to be a signal, swallowing its red defeats it. If it is meant to be informational only, it should not be a job that appears to gate the suite.
Source: QwenLM/qwen-code