Flaky test dashboard
Failures reported by the CI on main between 2026-09-10 and 2026-09-17 — 15 commits with CI results, 390 job executions.
Tests
| Test |
Occurrences |
Recovered on retry |
Commits affected |
Configurations |
Last seen |
TestSave 3 cases |
4 |
0 |
4/15 |
2 |
2026-09-17 |
TestBuildContextWithOCILayout |
3 |
0 |
3/15 |
3 |
2026-09-17 |
TestIPFSCompNoBuild |
3 |
3 |
3/15 |
1 |
2026-09-17 |
TestLoadStdinFromPipe |
3 |
0 |
2/15 |
3 |
2026-09-13 |
TestComposeStart |
2 |
0 |
2/15 |
1 |
2026-09-13 |
TestIPFSAddrWithKubo |
2 |
0 |
2/15 |
2 |
2026-09-16 |
TestRunHostLookup 8 cases |
2 |
0 |
2/15 |
2 |
2026-09-17 |
TestSaveMultipleImagesWithSameIDAndLoad 2 cases |
2 |
0 |
2/15 |
2 |
2026-09-17 |
TestStartWithCheckpoint |
2 |
0 |
2/15 |
2 |
2026-09-17 |
TestAttachForAutoRemovedContainer |
1 |
0 |
1/15 |
1 |
2026-09-13 |
TestComposeUpWithExternalNetwork |
1 |
0 |
1/15 |
1 |
2026-09-17 |
TestContainerListWithFilter 3 cases |
1 |
0 |
1/15 |
1 |
2026-09-17 |
TestLoadQuiet |
1 |
0 |
1/15 |
1 |
2026-09-10 |
TestLoginAgainstVariants 2 cases |
1 |
0 |
1/15 |
1 |
2026-09-17 |
TestPush 2 cases |
1 |
1 |
1/15 |
1 |
2026-09-17 |
TestRunAddHostRemainsWhenAnotherContainerCreated |
1 |
0 |
1/15 |
1 |
2026-09-11 |
TestRunDomainname 2 cases |
1 |
0 |
1/15 |
1 |
2026-09-14 |
TestRunInternetConnectivity 2 cases |
1 |
0 |
1/15 |
1 |
2026-09-13 |
TestRunRmTime |
1 |
0 |
1/15 |
1 |
2026-09-17 |
TestRunWithLogBinary |
1 |
0 |
1/15 |
1 |
2026-09-17 |
TestSaveContent |
1 |
0 |
1/15 |
1 |
2026-09-17 |
TestSaveQuiet |
1 |
0 |
1/15 |
1 |
2026-09-17 |
TestSave — 4 occurrences
| When |
Job configuration |
Test |
Outcome |
| 2026-09-17 |
in-host / nerdctl.gomodjail < rootless linux |
TestSave |
failed |
| 2026-09-17 |
in-host / nerdctl.gomodjail < rootless linux |
TestSave |
failed |
| 2026-09-17 |
in-host / nerdctl.gomodjail < rootless linux |
TestSave |
failed |
| 2026-09-17 |
in-host / rootless linux |
TestSave |
failed |
TestBuildContextWithOCILayout — 3 occurrences
| When |
Job configuration |
Test |
Outcome |
| 2026-09-17 |
in-host / rootful linux (arm) |
TestBuildContextWithOCILayout |
failed |
| 2026-09-17 |
in-host / rootless-port-slirp4netns linux |
TestBuildContextWithOCILayout |
failed |
| 2026-09-11 |
in-host / nerdctl.gomodjail < rootless linux |
TestBuildContextWithOCILayout |
failed |
TestIPFSCompNoBuild — 3 occurrences
| When |
Job configuration |
Test |
Outcome |
| 2026-09-17 |
in-host / rootful linux |
TestIPFSCompNoBuild |
recovered on retry |
| 2026-09-17 |
in-host / rootful linux |
TestIPFSCompNoBuild |
recovered on retry |
| 2026-09-17 |
in-host / rootful linux |
TestIPFSCompNoBuild |
recovered on retry |
TestLoadStdinFromPipe — 3 occurrences
| When |
Job configuration |
Test |
Outcome |
| 2026-09-13 |
in-host / nerdctl.gomodjail < rootless linux |
TestLoadStdinFromPipe |
failed |
| 2026-09-13 |
in-host / rootless-port-slirp4netns linux |
TestLoadStdinFromPipe |
failed |
| 2026-09-13 |
in-host / rootless linux |
TestLoadStdinFromPipe |
failed |
TestComposeStart — 2 occurrences
| When |
Job configuration |
Test |
Outcome |
| 2026-09-13 |
in-host / docker < linux |
TestComposeStart |
failed |
| 2026-09-11 |
in-host / docker < linux |
TestComposeStart |
failed |
TestIPFSAddrWithKubo — 2 occurrences
| When |
Job configuration |
Test |
Outcome |
| 2026-09-16 |
in-host / rootful linux (old ubuntu) (ctd: v1.7.35) |
TestIPFSAddrWithKubo |
failed |
| 2026-09-13 |
in-host / rootful linux |
TestIPFSAddrWithKubo |
failed |
TestRunHostLookup — 2 occurrences
| When |
Job configuration |
Test |
Outcome |
| 2026-09-17 |
in-host / nerdctl.gomodjail < rootless linux |
TestRunHostLookup |
failed |
| 2026-09-13 |
in-host / docker < linux |
TestRunHostLookup |
failed |
TestSaveMultipleImagesWithSameIDAndLoad — 2 occurrences
| When |
Job configuration |
Test |
Outcome |
| 2026-09-17 |
in-host / rootless linux (arm) |
TestSaveMultipleImagesWithSameIDAndLoad |
failed |
| 2026-09-17 |
in-host / nerdctl.gomodjail < rootless linux |
TestSaveMultipleImagesWithSameIDAndLoad |
failed |
TestStartWithCheckpoint — 2 occurrences
| When |
Job configuration |
Test |
Outcome |
| 2026-09-17 |
in-host / rootful linux (old ubuntu) (ctd: v1.7.35) |
TestStartWithCheckpoint |
failed |
| 2026-09-17 |
in-host / rootful linux (arm) |
TestStartWithCheckpoint |
failed |
TestAttachForAutoRemovedContainer — 1 occurrence
| When |
Job configuration |
Test |
Outcome |
| 2026-09-13 |
in-host / docker < linux |
TestAttachForAutoRemovedContainer |
failed |
TestComposeUpWithExternalNetwork — 1 occurrence
| When |
Job configuration |
Test |
Outcome |
| 2026-09-17 |
in-host / rootful linux (old ubuntu) (ctd: v1.7.35) |
TestComposeUpWithExternalNetwork |
failed |
TestContainerListWithFilter — 1 occurrence
| When |
Job configuration |
Test |
Outcome |
| 2026-09-17 |
in-host / nerdctl.gomodjail < rootless linux |
TestContainerListWithFilter |
failed |
TestLoadQuiet — 1 occurrence
| When |
Job configuration |
Test |
Outcome |
| 2026-09-10 |
in-host / nerdctl.gomodjail < rootless linux |
TestLoadQuiet |
failed |
TestLoginAgainstVariants — 1 occurrence
| When |
Job configuration |
Test |
Outcome |
| 2026-09-17 |
in-host / rootless-port-slirp4netns linux |
TestLoginAgainstVariants |
failed |
TestPush — 1 occurrence
| When |
Job configuration |
Test |
Outcome |
| 2026-09-17 |
in-host / rootful linux (old ubuntu) (ctd: v1.7.35) |
TestPush |
recovered on retry |
TestRunAddHostRemainsWhenAnotherContainerCreated — 1 occurrence
| When |
Job configuration |
Test |
Outcome |
| 2026-09-11 |
in-host / rootful linux (arm) |
TestRunAddHostRemainsWhenAnotherContainerCreated |
failed |
TestRunDomainname — 1 occurrence
| When |
Job configuration |
Test |
Outcome |
| 2026-09-14 |
in-host / nerdctl.gomodjail < rootless linux |
TestRunDomainname |
failed |
TestRunInternetConnectivity — 1 occurrence
| When |
Job configuration |
Test |
Outcome |
| 2026-09-13 |
EL / almalinux-8 rootless |
TestRunInternetConnectivity |
failed |
TestRunRmTime — 1 occurrence
| When |
Job configuration |
Test |
Outcome |
| 2026-09-17 |
in-host / nerdctl.gomodjail < rootless linux |
TestRunRmTime |
failed |
TestRunWithLogBinary — 1 occurrence
| When |
Job configuration |
Test |
Outcome |
| 2026-09-17 |
EL / almalinux-8 rootful |
TestRunWithLogBinary |
failed |
TestSaveContent — 1 occurrence
| When |
Job configuration |
Test |
Outcome |
| 2026-09-17 |
in-host / nerdctl.gomodjail < rootless linux |
TestSaveContent |
failed |
TestSaveQuiet — 1 occurrence
| When |
Job configuration |
Test |
Outcome |
| 2026-09-17 |
in-host / rootful linux |
TestSaveQuiet |
failed |
Job configurations
Every job, test or not: a job that fails without any test failing is usually an environment, timeout, or infrastructure problem.
| Job configuration |
Failure rate |
Failed |
Executions |
Recent failures |
in-host / nerdctl.gomodjail < rootless linux |
█████░░░░░ 53% |
8 |
15 |
1 2 3 |
in-host / windows |
█████░░░░░ 53% |
8 |
15 |
1 2 3 |
in-host / rootful linux (arm) |
██░░░░░░░░ 20% |
3 |
15 |
1 2 3 |
in-host / rootful linux (old ubuntu) (ctd: v1.7.35) |
██░░░░░░░░ 20% |
3 |
15 |
1 2 3 |
in-host / rootless-port-slirp4netns linux |
██░░░░░░░░ 20% |
3 |
15 |
1 2 3 |
in-host / docker < linux |
█░░░░░░░░░ 13% |
2 |
15 |
1 2 |
in-host / rootful linux |
█░░░░░░░░░ 13% |
2 |
15 |
1 2 |
in-host / rootless linux |
█░░░░░░░░░ 13% |
2 |
15 |
1 2 |
EL / almalinux-8 rootful |
█░░░░░░░░░ 7% |
1 |
15 |
1 |
EL / almalinux-8 rootless |
█░░░░░░░░░ 7% |
1 |
15 |
1 |
FreeBSD / FreeBSD |
█░░░░░░░░░ 7% |
1 |
15 |
1 |
in-host / rootless linux (arm) |
█░░░░░░░░░ 7% |
1 |
15 |
1 |
kubernetes |
█░░░░░░░░░ 7% |
1 |
15 |
1 |
windows-2022 | go |
█░░░░░░░░░ 7% |
1 |
15 |
1 |
How this is collected
This dashboard is generated by
Soigneur,
which aggregates the GitHub Actions annotations that the test jobs attach to themselves.
No database, test artifact, or server is involved, so the window that can be reported on is
limited by the retention of the check runs (90 days, by default).
- Occurrences: how many (commit × job configuration) executions reported the test as failing.
- Recovered on retry: occurrences where the test did pass when it was retried
(
gotestsum --rerun-fails). Those are flaky beyond a doubt. The others are either flaky,
or consistently broken.
- Only the integration suites report per-test data: the unit tests, Windows, and
FreeBSD do not run the reporter.
See https://github.com/containerd/nerdctl/blob/main/docs/testing/flaky.md to reproduce, fix, or quarantine a flaky test.
Generated by this run.