test: deflake sibling wall-clock perf-budget tests (pubmed / sqlcipher / alembic)
Author: LearningCircuitCreated Sep 19, 2026Updated Sep 19, 2026
PR #6623 fixed the wall-clock contention flake for the three scrub_error budget tests (test_log_sanitizer.py::TestScrubErrorUrlQueryRedaction) by measuring CPU time (time.process_time) — after #6612 and two later CI runs failed test_scrub_error_is_bounded_on_a_run_of_bare_schemes on wall-clock noise alone. The same fragility exists in sibling budget tests elsewhere in the suite, which will keep flaking on loaded xdist runners:
| Test | Budget | Note |
|---|---|---|
tests/research_library/downloaders/test_pubmed.py:332 |
~0.1s wall | near-zero headroom; one GC pause kills it — worst offender |
tests/security/test_sqlcipher_integration.py:1722 |
1.0s perf_counter over real SQLCipher I/O |
I/O-bound: cannot switch to process_time — needs a widened wall budget instead (see #6623's body for the factor analysis: core variance ×1.3, pollution ×1.7) |
tests/db/test_alembic_migrations.py (4 asserts) |
5–30s wall over real migration I/O | modest headroom on loaded CI |
Suggested treatment per class:
- CPU-bound budgets → switch to
time.process_time()and size the budget from measured healthy cost × worst-case compounding (see the convention note inTestScrubErrorUrlQueryRedaction's docstring). - I/O-bound budgets → keep the wall clock but widen with measured headroom; consider whether the budget guards a property that a size-invariant ratio (
t(2n) < 3·t(n)) would guard better.
Found during the 5-round review of #6623; a repo-wide sweep is git grep -nE 'assert.*(elapsed|took|duration).*<' -- tests/.
Source: LearningCircuit/local-deep-research