#4712·electric

Flaky timing-sensitive sync-service tests: ShapeCacheTest await/startup timeouts under CI load

Author: robacourtCreated Jul 14, 2026Updated Jul 14, 2026

Summary

A small family of timing-sensitive tests fails intermittently in the sync-service CI workflow — hopping between Postgres matrix versions, passing on rerun with no code change, and burying real signal for PR authors. Two recent PRs (#4711 and its review) each spent effort separating these from genuine failures.

Evidence

Electric.ShapeCacheTest — "await_snapshot_start/4 should wait for consumer to come up"

Electric.ShapeCacheTest — "get_or_create_shape_handle/2 against real db crashes when initial snapshot query fails to return data quickly enough"

Electric.Plug.TraceContextPlugTest — "electric member is found among other tracestate members"

ShapeLogCollectorTest — FlushTracker timing out in Repatch.cleanup

  • Reported during review of #4711 (first CI run of that branch)

Common signature

  • ~5s Task.await timeouts on tests that race consumer/snapshot startup
  • Postgres connection churn in the surrounding logs (tcp connect timeout, econnrefused, tcp recv (idle): closed) suggesting the shared PG instance is briefly saturated by concurrent async tests
  • Not reproducible locally on fast machines; version-hopping across the PG matrix

Suggested directions

  • Widen or parametrize the tight awaits in the affected tests (they're asserting eventual startup, not 5-second startup)
  • Or address the underlying PG saturation (connection pool sizing / readiness in the test env)
  • At minimum, tag the known-flaky set so authors and reviewers can distinguish them from real regressions without re-deriving this analysis

Filed while triaging CI on #4711, where these flakes twice muddied the signal.