pull-kubevirt-fuzz: SLOW timing assertion flake (seed 44, 798ms > 700ms)
/kind flake /sig compute
Saw this on my PR #19107 (commit b4c2419). That PR only deletes dormant code under pkg/virt-operator, but pull-kubevirt-fuzz failed with:
SLOW seed=44 case=SyntacticVirtualMachineFuzzing elapsed=798.86ms threshold=700ms
Looks like a runner-load flake, not a real regression:
- the 700ms limit is a hardcoded wall-clock check, pkg/virt-api/webhooks/fuzz/fuzz_test.go:143 (
timeoutDuration := 700 * time.Millisecond), so it mostly measures how busy the CI runner is at that moment - seeds 0..499 go into the fixed corpus via f.Add(int64(i)), so seed 44 runs on every job. SyntacticVirtualMachineFuzzing exercises the whole VMsAdmitter path incl. fake cluster config setup, which easily swings a few hundred ms on a loaded runner
- the PR can't be the trigger anyway, pkg/virt-api/webhooks/fuzz has no dependency on pkg/virt-operator (go list -deps)
Couldn't repro this locally, but that doesn't say much here since the package only builds on linux and the check is wall-clock based.
Job is non-required so this just drops a red X on random PRs and then a member has to /retest by hand (untrusted authors can't trigger jobs). For the record there was an earlier fuzz flake in #15973, but that one was a json panic, different failure.
Possible fixes:
- bump the threshold to something load-tolerant (2s-5s)
- or run each case a few times and assert on a median instead of a single sample
- or make it t.Logf only / move it to a benchmark job
Happy to put up a PR for whichever direction maintainers prefer.
Source: kubevirt/kubevirt