[Bug] Deployment analyzer reports in-progress rollouts as failures
Expected Behavior
A Deployment that is actively progressing (rollout or scaling in progress) should not be reported as a problem. status.readyReplicas trailing spec.replicas is a normal, healthy intermediate state while the Deployment controller is rolling out a new ReplicaSet or scaling up — image pulls and readiness probes take time, and maxSurge/maxUnavailable deliberately allow temporarily fewer ready pods.
Current Behavior
pkg/analyzer/deployment.go reports a failure whenever spec.replicas != status.readyReplicas, without inspecting the Deployment's progress state. During a normal rollout (e.g. spec.replicas: 3, status.replicas: 3, status.readyReplicas: 2), running k8sgpt analyze --filter Deployment emits:
Deployment default/web has 3 replicas but 2 are available with status runningThis is a false positive: the third pod is still pulling the image / waiting on its readiness probe, and the Deployment is healthy.
The check also does not consider status.observedGeneration vs metadata.generation. When the controller has not yet reconciled the latest spec (observedGeneration < generation), status is stale relative to spec, and the replica comparison is misleading. No analyzer in pkg/analyzer/ currently consults observedGeneration.
Steps to Reproduce
Reproduces purely at the status level with the fake client already used in pkg/analyzer/deployment_test.go (no live cluster required):
- Create a Deployment with
spec.replicas: 3and a pod template. - Set
status.replicas: 3,status.readyReplicas: 2,status.availableReplicas: 2— the mid-rollout shape. - Run the Deployment analyzer (
k8sgpt analyze --filter Deployment, or the unit test harness). - Observe the failure text above, even though nothing is wrong.
To reproduce on a live cluster: deploy a Deployment with a readiness probe, update its image, and run k8sgpt analyze during the rollout window (seconds to minutes depending on image size and probe time).
Environment
- k8sgpt version: main
- Kubernetes version: n/a (analyzer logic; reproduced with fake client). Rollout semantics are identical across versions.
- AI Backend/Provider: n/a (analyzer path,
--explainnot required) - OS/Platform: n/a
Additional Context
Current check:
if deployment.Spec.Replicas != nil && *deployment.Spec.Replicas != deployment.Status.ReadyReplicas {
// report replica mismatch only
}Suggested direction for the fix (open to maintainer preference; deliberately does not overlap with the Progressing=False / ProgressDeadlineExceeded reporting from #1739 / #1740, which covers the stuck rollout case):
ObservedGenerationguard: skip the replica comparison whilestatus.observedGeneration < metadata.generation(controller hasn't caught up to the latest spec yet).- Healthy-progress check: when
status.conditionsshowProgressing=True(withAvailable=True), treat the replica mismatch as in-progress and do not report; only report when the rollout is genuinely stuck or replicas are unavailable. - Add regression tests for: healthy rollout-in-progress, scaling-in-progress, controller-not-yet-synced (
observedGeneration < generation), and a genuinely stuck rollout (to keep the existing #1739 behavior).
Scope: single file pkg/analyzer/deployment.go + deployment_test.go.
I would like to work on this and can send a PR in that shape if the direction looks right.
Source: k8sgpt-ai/k8sgpt