#7054·angr

test_xchg_filler_decompiles_quickly_cross_insn_opt is flaky under Test with coverage: 60.15 s then 47.80 s on the same commit

Author: zardusCreated Sep 1, 2026Updated Sep 9, 2026

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

tests/analyses/decompiler/test_dead_assignment_removal.py::TestPackerFillerDecompilation::test_xchg_filler_decompiles_quickly_cross_insn_opt ends in assert elapsed < 60.0 after decompiling 5,000 blocks of xchg filler.

Same commit, same job, one fail and one pass

Run 33441452191, Test with coverage, shard Test (2), on commit 1562f5a1 of our open pull request #7053:

  • attempt 1 failed: AssertionError: decompiling 5000 blocks of filler took 60.1s, assert 60.14347696304321 < 60.0. The shard's test_durations.json recorded 60.153 s.
  • attempt 2, a re-run of that same job with no code change, passed at 47.799 s. 263 passed, 3 skipped, 0 failed.

The CI workflow on the same commit (33441452776) was green, and the same test passed there in job ci / Test (1).

Master's slowest run is 1.96 s under the bound

Of the 14 Test with coverage runs on master since the test landed in #7039 (2026-08-30T00:22Z), 8 produced a duration for it. The other 6 did not: four were cancelled before any Test job started, and in the other two the shard carrying this test was cancelled part-way through. Each figure below is read from that run's test_durations.json:

run commit seconds
33414732607 7421fa5fac 37.37
33284224031 3bd1578be5 41.99
33292646927 acfcdecb02 48.17
33416549670 6555a3f314 48.40
33332594613 4964e66dee 48.50
33403776145 078a172afa 48.77
33424730620 ace14d1bcd 48.96
33285200820 d82332d191 58.04

Eight observations, 37.37 s to 58.04 s. An unrelated branch, feature/mips-n32-cc run 33285256639, measured 57.09 s.

Runs 33414732607 and 33284224031 are marked cancelled, but all ten of their Test shards succeeded and only Report was cancelled, so the per-shard results-N artifact still carries the duration.

The bound has been raised once already

6cf759b0, "Loosen the time bounds.", took this bound from 30.0 to 60.0 and the sibling test_xchg_filler_decompiles_quickly from 8.0 to 20.0, after run 33259428096 on fix/issue-6968 measured 40.036 s and 10.129 s against the old bounds. Since then the slowest master run is the 58.04 s above, and the only measurement over 60.0 is the failed attempt on #7053.

Two options that do not use the clock

The one failure we have seen is in Test with coverage, which builds native code with CFLAGS: --coverage -O0 -g and RUSTFLAGS: -C instrument-coverage -C codegen-units=1 -C link-dead-code, and runs pytest under --cov=angr --cov=tests.

  • Skip the test when that instrumentation is on, leaving it as a timing check in CI, where it passed on the commit above.
  • Assert on a counted quantity rather than on elapsed time.

Your call which, if either. Happy to send the patch.

session: sharpen