pods/resize forbidden leads to pending state
What happened?
Hi,
In some shared cluster, it is not allowed to resize a pod. So the RBAC is not given to vcluster serviceaccount. But in recent kubernetes cluster (1.35+), the vcluster automatically assumes it has the permission to do resize: https://github.com/loft-sh/vcluster/blob/v0.35.2/pkg/controllers/resources/pods/syncer.go#L434
This leads to lots of error:
Error syncing: pods "git-shell-7d574f6646-jcmpk" is forbidden: User "system:serviceaccount:XXX:vc-my-vcluster" cannot patch resource "pods/resize" in API group "" in the namespace "XXX"Because the error is catched, then vcluster never ends the pod change of state from Pending to Ready.
What did you expect to happen?
A configuration to disable resize from vcluster, same behavior before this feature was implemented in vcluster.
How can we reproduce it (as minimally and precisely as possible)?
deploy vcluster, with helm values (see below) then deploy this deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: git-shell
spec:
replicas: 1
selector:
matchLabels:
app: git-shell
template:
metadata:
labels:
app: git-shell
spec:
containers:
- name: git
image: alpine/git:latest
command:
- sleep
- infinity
stdin: true
tty: trueAnything else we need to know?
No response
Host cluster Kubernetes version
1.35.5 Okd 4.22
$ kubectl version
Client Version: v1.32.3
Kustomize Version: v5.5.0
Server Version: v1.36.0vcluster version
$ vcluster --version
vcluster version 0.35.1VCluster Config
# My vcluster.yaml / values.yaml here
rbac:
# Role holds virtual cluster role configuration
role:
# Enabled defines if the role should be enabled or disabled.
enabled: true
# OpenShift has some RBAC limitation. Copy and adapt from https://github.com/loft-sh/vcluster/blob/v0.35.1/chart/templates/role.yaml#L29
overwriteRules:
- apiGroups: [""]
resources: ["configmaps", "secrets", "services", "pods", "pods/attach", "pods/portforward", "pods/exec", "persistentvolumeclaims"]
verbs: ["create", "delete", "patch", "update", "get", "list", "watch"]
- apiGroups: ["apps"]
resources: ["statefulsets", "replicasets", "deployments"]
verbs: ["get", "list", "watch", "patch", "update"]
- apiGroups: [""]
resources: ["endpoints", "pods/log"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: [ "create", "get", "list", "watch" ]
- apiGroups: ["discovery.k8s.io"]
resources: [ "endpointslices" ]
verbs: [ "list", "get", "watch" ]
- apiGroups: ["coordination.k8s.io"]
resources: ["leases"]
verbs: ["create", "delete", "patch", "update", "get", "list", "watch"]
- apiGroups: ["metrics.k8s.io"]
resources: ["pods"]
verbs: ["get", "list"]
- apiGroups: ["networking.k8s.io"]
resources: ["ingresses"]
verbs: ["create", "delete", "patch", "update", "get", "list", "watch"]
- apiGroups: ["gateway.networking.k8s.io"]
resources: ["gateways"]
#verbs: ["create", "delete", "patch", "update", "get", "list", "watch"]
verbs: ["get", "list", "watch"]
- apiGroups: ["gateway.networking.k8s.io"]
resources: ["httproutes"]
verbs: ["create", "delete", "patch", "update", "get", "list", "watch"]
- apiGroups: ["networking.k8s.io"]
resources: ["networkpolicies"]
verbs: ["create", "delete", "patch", "update", "get", "list", "watch"]
- apiGroups: [""]
resources: ["serviceaccounts"]
verbs: ["create", "delete", "patch", "update", "get", "list", "watch"]
- apiGroups: ["policy"]
resources: ["poddisruptionbudgets"]
verbs: ["create", "delete", "patch", "update", "get", "list", "watch"]
Additional info
there is at host a LimitRange with a default resource for containers deployed without resource. See https://github.com/loft-sh/vcluster/issues/4091#issuecomment-5003415919
Source: loft-sh/vcluster