flaky(e2e): oauth_auth "shows a specific message for oauthSignin error" retried on windows-latest
Caught incidentally while auditing CI logs for #3019 / #3021. The run reported green — this only surfaced by reading the raw shard logs.
What happened
Run 32954779496, attempt 3, shard e2e-tests / windows-latest-3 (job 98139320253):
(Attempt 1 of 4) shows a specific message for oauthSignin error 10:14:32.65
√ shows a specific message for oauthSignin error (1351ms) 10:14:34.09Spec: cypress/e2e/oauth_auth/spec.cy.ts. It failed once and passed on the retry, so retries: 3 absorbed it and the job finished All specs passed! — 43 passing, 0 failing.
Evidence it is a genuine flake
Cross-checked against 20 full shard logs (10 shards × attempts 2 and 3), plus attempt 1:
- The
(Attempt N of ...)annotation appears nowhere else in any log. - That job uniquely reports
Screenshots: 1, with... oauthSignin error (failed).png— a captured failure. - Its "Upload screenshots" step ran (
success); the same step isskippedon every other shard, i.e. nothing failed there. - The test normally completes in ~300–700ms wherever else it runs. This occurrence showed a 30s+ gap before the retry line, consistent with a command timing out rather than a fast assertion failure.
Only observed on windows-latest so far; ubuntu-latest-3 ran the same spec cleanly in all three attempts. One occurrence, so the rate is unknown.
Why it is worth filing despite being green
This is precisely the failure mode that makes flakiness accumulate unnoticed: a real intermittent failure occurred, CI reported success, and nothing anywhere surfaced it. Finding it required pulling and grepping 20 job logs by hand. See the companion issue on retry reporting.
Starting points
- The 30s gap points at a Cypress command timeout (
defaultCommandTimeout: 30000incypress.config.ts), so the likely shape is a wait on an element or route that did not appear, rather than a wrong assertion. - The failure screenshot is in the run artifacts (
cypress-screenshots-windows-latest-3), though artifact retention may expire it.
Source: Chainlit/chainlit