#3023·chainlit

flaky(e2e): oauth_auth "shows a specific message for oauthSignin error" retried on windows-latest

Author: dokterbobCreated Aug 26, 2026Updated Aug 26, 2026
Labelsbugneeds-triagee2e-tests

Caught incidentally while auditing CI logs for #3019 / #3021. The run reported green — this only surfaced by reading the raw shard logs.

What happened

Run 32954779496, attempt 3, shard e2e-tests / windows-latest-3 (job 98139320253):

(Attempt 1 of 4) shows a specific message for oauthSignin error     10:14:32.65
√ shows a specific message for oauthSignin error (1351ms)           10:14:34.09

Spec: cypress/e2e/oauth_auth/spec.cy.ts. It failed once and passed on the retry, so retries: 3 absorbed it and the job finished All specs passed! — 43 passing, 0 failing.

Evidence it is a genuine flake

Cross-checked against 20 full shard logs (10 shards × attempts 2 and 3), plus attempt 1:

  • The (Attempt N of ...) annotation appears nowhere else in any log.
  • That job uniquely reports Screenshots: 1, with ... oauthSignin error (failed).png — a captured failure.
  • Its "Upload screenshots" step ran (success); the same step is skipped on every other shard, i.e. nothing failed there.
  • The test normally completes in ~300–700ms wherever else it runs. This occurrence showed a 30s+ gap before the retry line, consistent with a command timing out rather than a fast assertion failure.

Only observed on windows-latest so far; ubuntu-latest-3 ran the same spec cleanly in all three attempts. One occurrence, so the rate is unknown.

Why it is worth filing despite being green

This is precisely the failure mode that makes flakiness accumulate unnoticed: a real intermittent failure occurred, CI reported success, and nothing anywhere surfaced it. Finding it required pulling and grepping 20 job logs by hand. See the companion issue on retry reporting.

Starting points

  • The 30s gap points at a Cypress command timeout (defaultCommandTimeout: 30000 in cypress.config.ts), so the likely shape is a wait on an element or route that did not appear, rather than a wrong assertion.
  • The failure screenshot is in the run artifacts (cypress-screenshots-windows-latest-3), though artifact retention may expire it.