runtime: a hugepage-backed guest ignores the pod's hugepage declaration

Update 2026-09-18. #13542 was narrowed to the vCPU pinning fix, so the memory half is back here. A draft PR (#13853) implements the proposals below, reading the reservation from the pod's cgroup because the CRI does not carry it (kubernetes/enhancements#4113). Reproduced on 4.1.0 and main.

Two things have moved since this was filed.

Proposal 1 below says default_memory would be the fallback when a pod declares no huge pages. That branch turned out to be unreachable. The kubelet writes hugetlb.<size>.max = 0 for a pod that declares none, and against a limit of zero a huge page backed VM cannot map a single page, whatever default_memory says. Such a pod is refused instead, with a message naming hugepages-<size> and the runtime class's pod overhead. default_memory is not a floor under a pod that reserves less, either: the key need not be in the configuration at all, in which case the runtime has already substituted its own default and cannot tell the two apart.

The work also grew past the two proposals, because sizing the VM from the pod's reservation settles only half of the problem. Once the reservation is the guest, three more questions have to be answered: what a container's hugepages-<size> means inside the guest, what is left for memory to mean, and where the guest's own share of its RAM comes from. The design I ended up with, the pod shape I would recommend to an end user, and a case matrix of main against the branch are in the comment below. I would rather hear that the shape is wrong now than after it is written twice.

The draft carries @fidencio's runtime-rs commits unchanged and none of my own: runtime-rs is better written once the shape below is agreed than twice.

Which feature do you think can be improved?

Guest memory sizing for hugepage-backed sandboxes (enable_hugepages = true) under static_sandbox_resource_mgmt = true. Split out of https://github.com/kata-containers/kata-containers/issues/13539#issuecomment-5192940339. #13542 settles what static sizing does with a memory limit on such a guest; this is about what is left after it: the guest size still lives in two places that nothing reconciles, and the two sizing paths disagree about which place wins.

On 4.0.0 (cf82bb35):

• Static sizing: the guest is sized from the runtime class's default_memory. updateResources() returns before any per-container accounting (sandbox.go#L2458), so the pod's hugepages-1Gi limit never reaches the VM size. With #13542 this becomes the stated contract: default_memory alone.
• Hotplug sizing: calculateSandboxMemory() folds the container HugepageLimits into the VM size (sandbox.go#L2660, from kata-containers/runtime#3109). The pod declaration wins.

So the same pod spec means two different guests depending on one runtime toggle, and on the static path, running a hugepage-backed guest requires stating its size twice:

• default_memory in the runtime class, which the runtime uses;
• limits.hugepages-<size> in the pod, which the kubelet/scheduler use to reserve pages from the node pool.

Nothing checks them against each other. The failure when they drift is not attributed to either knob: QEMU dies at prealloc with

qemu-system-x86_64: unable to map backing store for guest RAM: Cannot allocate memory

which reads as node memory pressure, not as "the pod reserved fewer pages than default_memory needs". We have seen this exact message from three unrelated causes: the declaration drift above, the hugetlb pool still draining a previous guest's pages when the next one preallocates (transient, resolved by the kubelet's retry), and a host kernel VFIO regression. Only the first is a misconfiguration. Nothing in the message says which one it was, and each time the answer came from the QEMU log in the node journal.

A second-order cost: one runtime class per guest size. Since the size is fixed in default_memory, every VM shape needs its own runtime class + configuration drop-in (we carry four, one per GPU tier). The pod already declares the exact same quantity in hugepages-<size>; the class copy exists only because the runtime does not read the pod's.

How can it be improved?

What the linked comment sketched, which matches what we would want as operators:

1. For a hugetlb-backed guest under static sizing, let the pod's limits.hugepages-<size> be the guest size, with default_memory as the fallback when the pod declares none. The size is then declared once, in the pod; one runtime class serves every size; and the only remaining failure is the scheduler refusing a reservation the pool cannot meet, which names the right cause at the right layer.
2. Independently, and cheap on both runtimes: at VM start, compare the guest RAM about to be allocated against the sandbox's hugetlb allowance, and fail with both numbers when the allowance is smaller. That turns the QEMU prealloc death above into an error that says which knob is at fault, and separates a real misconfiguration from the transient and kernel-side causes that share the same QEMU message today.

The first changes what an existing pod field means on one path, so it probably needs a config gate or at least a release note. The second changes no behavior and would have saved us the journal round trip each time.

Additional Information

Out of scope here: the hotplug path's opposite convention stays as it is (per the discussion in #13539), and the vCPU side has an analogous two-source shape (default_vcpus boots on top of limits.cpu) that was already set aside for its own discussion in the same comment linked above. This issue keeps to hugepage-backed memory.

Environment: Kata 4.0.0 (cf82bb35), go runtime, QEMU, static_sandbox_resource_mgmt = true, enable_hugepages = true. Guests up to 2688 GiB backed by boot-time 1GiB hugepages; the pool takes nearly all node RAM.

Before raising this enhancement request

Have you looked at the limitations document?

• Yes