Verified: 0e107a38 (fs/zfs VFS fallback) fixes a fatal kubelet crash on the ZFS graph driver, but is unreleased — request a release

Summary

Commit 0e107a38 ("fs/zfs: fall back to VFS when zfs stats are unavailable", part of the #3918 fix) is on master but is not in any tagged release (git tag --contains 0e107a38 is empty; it landed after v0.60.5). Because kubelet vendors a released cadvisor, downstreams still hit a fatal kubelet startup crash on ZFS-backed nodes. Requesting a release be cut so the fix is consumable.

Why it matters (the failure the fix closes)

On a Kubernetes node whose container store uses the podman/containerd ZFS graph driver (e.g. a kind node run under podman with storage.driver = "zfs"), /dev/zfs is present in the node but the zfs CLI is not installed. cadvisor's GetZfsStats shells out to zfs list ..., which fails with exec ENOENT (exit 127).

In released cadvisor (v0.56.2, as vendored by kubelet v1.36.1 — and unchanged through v0.60.5), fs/zfs/plugin.go GetStats does:

capacity, free, avail, err := getZfsStats(device)
if err != nil {
    return nil, err          // <-- no VFS fallback
}

So no Fs is produced for the ZFS rootfs partition, and the kubelet dies:

container_manager_linux.go "Unable to get rootfs data from cAdvisor interface"
  err="cannot find filesystem info for device \"tank/podman/kind-probe/graph\""
kubelet.go "Failed to start ContainerManager"
  err="failed to get rootfs info: cannot find filesystem info for device ..."

ContainerManager never starts, kubelet exits 1, the API server never comes up.

master (0e107a38) changes that error path to return nil, fs.ErrFallbackToVFS, which routes to the VFS/statfs plugin and yields valid stats — but it is unreleased.

Verification the fix closes it

Checked out at cmosetick/cadvisor (verbatim master, HEAD 6a0c4f25):

• lib/fs/zfs/plugin_test.go TestGetStatsFallsBackOnZfsError passes — it asserts exactly our node condition (/dev/zfs present + getZfsStats error -> ErrFallbackToVFS).
• Added an end-to-end test driving GetFsInfoForPath with a ZFS partition whose plugin returns ErrFallbackToVFS; master produces a valid Fs via statfs fallback (non-zero Capacity). On v0.56.2/v0.60.5 the same input yields no Fs -> the "cannot find filesystem info for device" crash.

Full evidence writeup (node output, v0.56.2-vs-master diff, repro) and the end-to-end test source: https://gist.github.com/cmosetick/cd722e7f5897348046778432c19095db

Environment

• released cadvisor affected: v0.56.2 (vendored by kubelet v1.36.1 / kindest/node:v1.36.1); same code through v0.60.5.
• fix commit: 0e107a38 on master, in no tag.
• kind: v0.32.0, KIND_EXPERIMENTAL_PROVIDER=podman
• podman: 6.0.2 (rootful), storage driver = zfs
• host ZFS: zfs-2.2.2, kernel 6.8.0-136-generic

Ask

Cut a tagged release including 0e107a38 so kubelet/kind can vendor a cadvisor that no longer crashes on ZFS graph-driver nodes lacking the zfs CLI.

Related: #3904, #3918 (the fix), #3832/#3879 (separate ZFS double-count, still present in stats.go).