test_xchg_filler_decompiles_quickly_cross_insn_opt is flaky under Test with coverage: 60.15 s then 47.80 s on the same commit
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS
tests/analyses/decompiler/test_dead_assignment_removal.py::TestPackerFillerDecompilation::test_xchg_filler_decompiles_quickly_cross_insn_opt
ends in assert elapsed < 60.0
after decompiling 5,000 blocks of xchg filler.
Same commit, same job, one fail and one pass
Run 33441452191, Test with coverage, shard Test (2), on commit 1562f5a1 of our open pull request #7053:
- attempt 1 failed:
AssertionError: decompiling 5000 blocks of filler took 60.1s,assert 60.14347696304321 < 60.0. The shard'stest_durations.jsonrecorded 60.153 s. - attempt 2, a re-run of that same job with no code change, passed at 47.799 s. 263 passed, 3 skipped, 0 failed.
The CI workflow on the same commit (33441452776) was green, and the same test passed there in job ci / Test (1).
Master's slowest run is 1.96 s under the bound
Of the 14 Test with coverage runs on master since the test landed in #7039 (2026-08-30T00:22Z), 8 produced a duration for it. The other 6 did not: four were cancelled before any Test job started, and in the other two the shard carrying this test was cancelled part-way through. Each figure below is read from that run's test_durations.json:
| run | commit | seconds |
|---|---|---|
| 33414732607 | 7421fa5fac | 37.37 |
| 33284224031 | 3bd1578be5 | 41.99 |
| 33292646927 | acfcdecb02 | 48.17 |
| 33416549670 | 6555a3f314 | 48.40 |
| 33332594613 | 4964e66dee | 48.50 |
| 33403776145 | 078a172afa | 48.77 |
| 33424730620 | ace14d1bcd | 48.96 |
| 33285200820 | d82332d191 | 58.04 |
Eight observations, 37.37 s to 58.04 s. An unrelated branch, feature/mips-n32-cc run 33285256639, measured 57.09 s.
Runs 33414732607 and 33284224031 are marked cancelled, but all ten of their Test shards succeeded and only Report was cancelled, so the per-shard results-N artifact still carries the duration.
The bound has been raised once already
6cf759b0, "Loosen the time bounds.", took this bound from 30.0 to 60.0 and the sibling test_xchg_filler_decompiles_quickly from 8.0 to 20.0, after run 33259428096 on fix/issue-6968 measured 40.036 s and 10.129 s against the old bounds. Since then the slowest master run is the 58.04 s above, and the only measurement over 60.0 is the failed attempt on #7053.
Two options that do not use the clock
The one failure we have seen is in Test with coverage, which builds native code with CFLAGS: --coverage -O0 -g and RUSTFLAGS: -C instrument-coverage -C codegen-units=1 -C link-dead-code, and runs pytest under --cov=angr --cov=tests.
- Skip the test when that instrumentation is on, leaving it as a timing check in
CI, where it passed on the commit above. - Assert on a counted quantity rather than on elapsed time.
Your call which, if either. Happy to send the patch.
session: sharpen
Source: angr/angr