#10875·zeroclaw

[Bug]: Flaky: Telegram media-group tests fail the Parallel Runtime Test repeat step on unrelated PRs

Author: JordanTheJetCreated Sep 15, 2026Updated Sep 17, 2026
Labelsbugcichanneltestschannel:telegrampriority:p1status:in-progressstatus:accepted

Affected component

channel

Severity

S1 - workflow blocked

Current behavior

Since cbd1b0adce (#8955, merged 2026-09-11) the Parallel Runtime Test job fails intermittently on PRs that do not touch crates/zeroclaw-channels/src/telegram.rs, which turns CI Required Gate red and blocks merge. The failure is always a single Telegram media-group test out of ~1590 in zeroclaw-channels --lib, on one of the three 16-thread repeat runs, and a gh run rerun --failed at the same head passes.

Four tests from the same family have hit so far:

Test Assertion Seen on
media_group_listener_retains_video_context_before_photos_across_polls telegram.rs:12058 .expect("album should dispatch") on a 4 s timeout #10637 run 34622512730, #10664 run 34876710466 (both reruns passed)
media_group_holds_across_saturated_page_ending_in_an_ordinary_update telegram.rs:12561 .expect("page-boundary message should dispatch") on a 2 s timeout #10824 run 34765230979, while #10823 with the identical runtime change passed
media_group_stays_pending_when_a_later_unsupported_member_follows_an_ordinary_update timeout waiting for album dispatch #9809
media_group_trailing_saturated_by_page_boundary_waits_for_next_page telegram.rs:12892 timeout #10401, run 3/3

Cause, from reading the tests: each drives sequential getUpdates polls through a wiremock responder that adds a 750 ms delay once its scripted updates are drained, then asserts dispatch inside fixed 1 s / 2 s / 4 s tokio::time::timeout guards. On the 16-thread repeat step that scheduling slack is not always available. This is the same shape #10254 fixed for the seventeen Telegram listen tests.

Expected behavior

The media-group tests pass deterministically under scripts/ci/parallel_runtime_test_gate.sh, so an unrelated PR's required gate reflects that PR's change. Proposed fix: drive the tests on a paused tokio clock with explicit advance, or await an explicit dispatch signal from the batching path, instead of wall-clock guards, while keeping the saturation coverage so the guards can still catch a real regression.

Steps to reproduce

bash
# What CI runs (job "Parallel Runtime Test", step "Repeat parallel runtime tests"):
./scripts/ci/parallel_runtime_test_gate.sh
# equivalent to, three times per crate:
cargo test --locked --quiet -p zeroclaw-channels --lib -- --test-threads=16

# Intermittent: fails roughly one run in several on a loaded ubuntu-latest runner.
# Focused local reruns of the single test pass, so a contended host is part of the recipe:
cargo test -p zeroclaw-channels --lib -- --test-threads=16 telegram::tests::media_group_

Impact

Affected users: every PR author, most visibly external contributors whose approved PRs sit red on a test they did not touch (#10637, #10664, #10824, #9809, #10401 in four days). Frequency: intermittent, several hits per day across the PR queue. Consequence: CI Required Gate red until a maintainer reruns the job; maintainers have to attribute each red gate by hand before merging.

Workaround until fixed: gh run rerun <run-id> --failed on the affected PR. Do not ask contributors to rebase or force-push for this.

Logs / stack traces

==> parallel runtime regression: zeroclaw-channels run 3/3 (16 threads)
failures:

---- telegram::tests::media_group_listener_retains_video_context_before_photos_across_polls stdout ----

thread 'telegram::tests::media_group_listener_retains_video_context_before_photos_across_polls' (35628) panicked at crates/zeroclaw-channels/src/telegram.rs:12058:14:
album should dispatch: Elapsed(())


failures:
    telegram::tests::media_group_listener_retains_video_context_before_photos_across_polls

test result: FAILED. 1589 passed; 1 failed; 2 ignored; 0 measured; 0 filtered out; finished in 11.39s

error: test failed, to rerun pass `-p zeroclaw-channels --lib`

ZeroClaw version

master, observed at 003b4509d1 (PR #10664 head, 2026-09-14) and 8937f1ea10 (PR #10637 head, 2026-09-11); test family introduced in cbd1b0adce

Rust version

rustc 1.98.0 (88d9e12ae 2026-08-18), the CI stable toolchain

Operating system

ubuntu-latest GitHub-hosted runner (Ubuntu 24.04); not reproduced on macOS locally

Regression?

Yes, it worked before

Pre-flight checks

  • I reproduced this on the latest master branch or latest release.
  • I redacted secrets, tokens, and personal data from all submitted content.