#142145 Native sidecar container which is not in a ready state should not affect the pod readiness, atleast it should be configurable

What happened?

As of now controller manager checks the status of both containers and initcontainers with restartPolicy:Always( will refer as Native Side car) . Whenever any of the container is not in a ready state, this is also reflected in the pod status , setting pod.conditions.[type="Ready"] to false. This means when a side car container is not in a ready state, pod is also not deemed to be ready and the EndpointSlices which checks for status field of pod.conditions.[type="Ready"] also stops serving traffic to the main application. This is also confirmed in the KEP proposal Readiness probes of sidecars will contribute to determine the whole Pod readiness.

I believe the above condition needs to be evaluated again. We need to consider the readiness of a regular container(spec.containers) different from a side car in spirit of not using the old pattern :slightly_smiling_face: .

Let’s take some real world examples where the current behavior fails:

I have a log collector side car and even when the side car gets killed or has an error, traffic to the main application should not be stopped.
I have a backup sidecar container which takes backup of data, however when the backup container gets OOM killed, traffic to main application container still needs to be served

Cases where current pattern might be helpful

Service mesh

Is there any reason for enforcing this pattern ? In an ideal world, there could be a field to define the readiness behavior of the pod from the sidecar level, but this might be an overkill. Atleast , ideally readiness of a pod should not be impacted by side car readiness. There are some tweaks which we can do with

What did you expect to happen?

Side car getting killed should not set pod ready status to False and main application container should still serve the traffic.

How can we reproduce it (as minimally and precisely as possible)?

Create a nginx container and a native side car with service on top of it.

apiVersion: v1
kind: Pod
metadata:
  name: nginx-with-oom-sidecar
  labels:
    app: nginx-with-oom-sidecar

spec:
  restartPolicy: Always
  initContainers:
    - name: oom-sidecar
      image: python:3.12-alpine
      restartPolicy: Always  # Sidecar
      command:
        - /bin/sh
        - -c
        - |
          echo "Native sidecar started."
          echo "Sleeping before triggering OOM..."
          sleep 60

          echo "Attempting to allocate 256 MiB and trigger OOM"
          python3 -c '
          import time
          x = bytearray(256 * 1024 * 1024)
          print("Allocated memory")
          time.sleep(300)
          '

      resources:
        requests:
          cpu: 10m
          memory: 16Mi
        limits:
          cpu: 100m
          memory: 32Mi

  containers:
    - name: nginx
      image: nginx:1.27-alpine

      ports:
        - name: http
          containerPort: 80
      resources:
        requests:
          cpu: 50m
          memory: 32Mi
        limits:
          cpu: 200m
          memory: 128Mi
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-with-oom-sidecar
spec:
  selector:
    app: nginx-with-oom-sidecar
  ports:
    - name: http
      port: 80
      targetPort: 80
  type: ClusterIP

Create another pod to run curl command and hit the service

kubectl run curl-test \
  --image=curlimages/curl \
  --restart=Never \
  -it --rm \
  -- sh

Inside the container keep hitting the service

~ $ while true; do curl nginx-with-oom-sidecar ; sleep 2 ; done

Output(trimmed for readability) will be something similar to this, it will flip from reaching nginx to unable to reach nginx

<!DOCTYPE html>
<html>
<head>
...
<!DOCTYPE html>
<html>
<head>
...
<!DOCTYPE html>
<html>
<head>
...
curl: (7) Failed to connect to nginx-with-oom-sidecar:80 after 1 ms: Could not connect to server
curl: (7) Failed to connect to nginx-with-oom-sidecar:80 after 1 ms: Could not connect to server
...
and the scenario repeats

Anything else we need to know?

No response

Kubernetes version

Any version ```console % kubectl version Client Version: v1.36.0 Kustomize Version: v5.8.1 Server Version: v1.36.2+k3s1 ```

Native sidecar container which is not in a ready state should not affect the pod readiness, atleast it should be configurable

What happened?

What did you expect to happen?

How can we reproduce it (as minimally and precisely as possible)?

Anything else we need to know?

Kubernetes version

Cloud provider

OS version

Install tools

Container runtime (CRI) and version (if applicable)

Related plugins (CNI, CSI, ...) and versions (if applicable)