SELinux enforcing: bundled local-path helper cannot delete MCS-labeled PV directories
Environmental Info: K3s Version:
k3s version v1.36.2+k3s1 (01b6f04a)
go version go1.26.4The behavior was also verified against K3s main at commit 866d743e56ecec4a7a9860a22f9930afa39ff722, whose bundled local-storage manifest uses local-path-provisioner v0.0.36 and still has no SELinux options in helperPod.yaml.
Node(s) CPU architecture, OS, and Version:
SUSE Linux Enterprise Server 16.0
Linux <hostname-redacted> 6.12.0-160000.5-default x86_64 GNU/Linux- SELinux targeted policy, enforcing
- MLS enabled
k3s-selinux-1.6-1.slemicro.noarchinstalledsemodule -lreports thek3smodule loaded- containerd
2.3.2-k3s2
Cluster Configuration:
Three server nodes and one agent. K3s is configured with selinux: true on all nodes and uses the default bundled local-path StorageClass.
Describe the bug:
With SELinux enforcing, a PVC using the bundled local-path provisioner can be created and used successfully, but its PV and backing directory cannot be deleted.
The create helper and later delete helper receive different MCS category pairs. The volume directory keeps the categories assigned during creation, so the delete helper is denied access to it. The PVC/namespace deletion request succeeds, but the PV remains Released, VolumeFailedDelete repeats, and the backing directory leaks on the node.
This is a recurrence/silent regression of #10130. That issue was closed after rancher/local-path-provisioner#402 fixed the mismatch by assigning the full category range to the helper. The fix was subsequently reverted by rancher/local-path-provisioner#421 because hard-coding the value in Go overrode user-provided helper templates. The K3s issue was not reopened, and the current K3s bundled helper template still does not supply an equivalent default.
Steps To Reproduce:
- Installed K3s:
curl -sfL https://get.k3s.io | \
INSTALL_K3S_VERSION='v1.36.2+k3s1' \
INSTALL_K3S_EXEC='server --selinux' sh -
getenforce
semodule -l | grep '^k3s'
rpm -q k3s-selinuxConfirm that SELinux reports Enforcing and that the K3s policy is loaded.
- Apply the following resources:
apiVersion: v1
kind: Namespace
metadata:
name: selinux-probe
labels:
pod-security.kubernetes.io/enforce: privileged
pod-security.kubernetes.io/audit: privileged
pod-security.kubernetes.io/warn: privileged
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: probe-pvc
namespace: selinux-probe
spec:
accessModes:
- ReadWriteOnce
storageClassName: local-path
resources:
requests:
storage: 16Mi
---
apiVersion: v1
kind: Pod
metadata:
name: probe
namespace: selinux-probe
spec:
containers:
- name: probe
image: rancher/mirrored-library-busybox:1.37.0
command: ['sh', '-c', 'echo probe >/data/probe; sleep 3600']
volumeMounts:
- name: data
mountPath: /data
volumes:
- name: data
persistentVolumeClaim:
claimName: probe-pvc- Wait for the pod, then record the PV and backing path:
kubectl -n selinux-probe wait --for=condition=Ready pod/probe --timeout=120s
PV_NAME=$(kubectl -n selinux-probe get pvc probe-pvc -o jsonpath='{.spec.volumeName}')
VOLUME_PATH=$(kubectl get pv "$PV_NAME" -o jsonpath='{.spec.local.path}')
echo "$PV_NAME $VOLUME_PATH"- Delete the namespace and watch the PV/helper:
kubectl delete namespace selinux-probe
kubectl get pv "$PV_NAME" -w
kubectl -n kube-system get pods | grep helper-pod-delete
kubectl get events -A --sort-by=.lastTimestamp | grep VolumeFailedDeleteExpected behavior:
The delete helper succeeds. The PV object and backing directory are removed while SELinux remains enforcing.
Actual behavior:
The PV remains Released; the delete helper exits with status 1; and the controller retries every approximately 120 seconds:
VolumeFailedDelete: failed to delete volume ...:
create process timeout after 120 secondsThe reproduced directory and helper had different MCS categories:
volume target: system_u:object_r:container_file_t:s0:c432,c816
delete helper: system_u:system_r:container_t:s0:c471,c511When a retry was observed in permissive mode, the kernel recorded the operations that enforcing prevented:
avc: denied { write } for comm="rm"
scontext=system_u:system_r:container_t:s0:c471,c511
tcontext=system_u:object_r:container_file_t:s0:c432,c816
tclass=dir permissive=1
avc: denied { remove_name } for comm="rm"
scontext=system_u:system_r:container_t:s0:c471,c511
tcontext=system_u:object_r:container_file_t:s0:c432,c816
tclass=dir permissive=1
avc: denied { unlink } for comm="rm"
scontext=system_u:system_r:container_t:s0:c471,c511
tcontext=system_u:object_r:container_file_t:s0:c432,c816
tclass=file permissive=1The PV and directory disappeared on that retry. SELinux was immediately restored to enforcing.
Additional context / logs:
Prior reports and fix history:
- k3s-io/k3s#10130 reports the same
ReleasedPV, failed delete helper, permissive-mode success, and MCS mismatch. - rancher/local-path-provisioner#402 originally set helper containers to
s0-s0:c0.c1023; this shipped in v0.0.27. - rancher/local-path-provisioner#421 reverted the hard-coded Go change because it overrode the configurable helper template.
- rancher/local-path-provisioner#484 and k3s-io/k3s-selinux#52 report the same class of cross-MCS cleanup failure; neither resulted in a current default fix.
The supported helper-template mitigation was validated on the preserved enforcing cluster:
helperPod.yaml: |-
apiVersion: v1
kind: Pod
metadata:
name: helper-pod
spec:
containers:
- name: helper-pod
image: rancher/mirrored-library-busybox:1.37.0
imagePullPolicy: IfNotPresent
securityContext:
seLinuxOptions:
level: s0-s0:c0.c1023Results while SELinux remained enforcing:
- An existing PV that had been stuck with
VolumeFailedDeletefor more than 37 minutes was deleted within 15 seconds after patching the ConfigMap and restarting the provisioner. - A fresh PVC/pod lifecycle became ready in about 20 seconds, and its PV disappeared less than 10 seconds after namespace deletion.
Putting the default in K3s's bundled helperPod.yaml would avoid the specific objection that caused #402 to be reverted: local-path-provisioner would continue honoring the configured template instead of overriding it in Go. An equivalent solution that safely preserves/reuses the volume's MCS label would also address the problem.
The current K3s tests appear not to catch the leak: the E2E test verifies volume use but does not delete the PVC, while the integration test checks that kubectl delete pvc returns successfully but does not wait for the PV object and backing directory to disappear.
Source: k3s-io/k3s