`helm uninstall --wait` with kstatus watcher times out on resources deletion
Helm version
4.1.0Kubernetes version
K3s v1.31.x (also reproducible on other distributions)Describe the bug
After upgrading to Helm 4.1.0, our integration tests running on GitHub Actions workflows started failing intermittently during helm uninstall.
When using helm uninstall --wait --cascade foreground, the new kstatus-based StatusWatcher (default in Helm 4.1.0) intermittently reports actualStatus=Terminating indefinitely for resources (ClusterRole, ClusterRoleBinding, Services and ServiceAccounts), even after Kubernetes has successfully deleted them.
This is a flaky issue - it does not occur on every uninstall, but happens frequently enough to cause CI failures. The issue appears to be a race condition in how the kstatus watcher tracks deletion status of resources during foreground cascade deletion. This issue did not occur with Helm 4.0.x.
When it occurs, the Kubernetes garbage collector removes the foregroundDeletion finalizer and deletes the resources within milliseconds, but Helm's status watcher never detects the deletion and waits until the timeout expires.
Using --wait=legacy works correctly and does not exhibit this flaky behavior.
This might be related to https://github.com/helm/helm/pull/31421 which introduced the new --wait flag strategies in Helm 4.1.0.
To Reproduce
- Install a Helm chart that creates resources (ClusterRole, ClusterRoleBinding, Service, ServiceAccount)
- Run
helm uninstall <release> --wait --cascade foreground --debug - Repeat steps 1-2 multiple times (the issue is intermittent)
- Observe Helm occasionally waiting for resources that are already deleted
We have integration tests running on GitHub Actions workflows that intermittently reproduce this issue:
- Repository: https://github.com/hivemq/helm-charts
- Test location:
tests-hivemq-platform-operator/src/integrationTest/java/com/hivemq/helmcharts/ - Chart:
charts/hivemq-platform-operator(creates ClusterRole, ClusterRoleBinding, ServiceAccount, Service) - Environment: K3s running inside Testcontainers
Expected behavior
helm uninstall --wait should detect that resources have been deleted and complete successfully.
Actual behavior
Helm logs show repeated polling with actualStatus=Terminating:
level=DEBUG msg="waiting for resources to be deleted" count=6 timeout=3m0s
level=DEBUG msg="waiting for resource" namespace="" name=hivemq-platform-operator-role-test-hivemq-platform-operator kind=ClusterRole expectedStatus=NotFound actualStatus=Terminating
level=DEBUG msg="waiting for resource" namespace="" name=hivemq-platform-operator-role-test-hivemq-platform-operator kind=ClusterRole expectedStatus=NotFound actualStatus=Terminating
...Meanwhile, Kubernetes garbage collector logs show successful deletion (~200ms after uninstall started):
[garbagecollector.go:657] "remove DeleteDependents finalizer for item" item="[rbac.authorization.k8s.io/v1/ClusterRole, namespace: , name: hivemq-platform-operator-role-test-hivemq-platform-operator, uid: b17c3762-8f22-4afa-a965-b22c1691fdda]"Workaround
Use --wait=legacy to revert to the original waiting mechanism:
helm uninstall <release> --wait=legacy --cascade foregroundAdditional context
- The issue is flaky/intermittent, likely due to a race condition in the kstatus watcher
- The issue appears specific to cluster-scoped resources (ClusterRole, ClusterRoleBinding)
- Namespaced resources in the same chart are handled correctly
- Potentially related PR that introduced the new wait strategies: https://github.com/helm/helm/pull/31421
Source: helm/helm