nested-virt vIOMMU sidecar hook + 2 VFIO passthrough devices fails to start

Title

VM with nested-virt vIOMMU sidecar hook + 2 VFIO passthrough devices fails to start: vfio_container_dma_map ENOMEM (RLIMIT_MEMLOCK), regardless of KubeVirt version or host memory available

What happened

A VMI with two gpus/passthrough devices, combined with an onDefineDomain hook sidecar that injects an emulated vIOMMU (-device intel-iommu,intremap=on,caching-mode=on,eim=on + -machine kernel-irqchip=split) to support nested virtualization / L1→L2 VFIO passthrough, fails to start with:

internal error: QEMU unexpectedly closed the monitor: ...
vfio 0000:xx:00.0: failed to setup container for group N: memory listener
initialization failed: Region ram-node0: vfio_container_dma_map(...) = -12
(Cannot allocate memory)

This occurs regardless of:

• Guest memory size (reproduced at both 96Gi/hugepages-1Gi and 8Gi/no-hugepages)
• Host memory availability (756Gi total, 150Gi+ free on the affected host at time of failure — this is not a real memory shortage)
• KubeVirt version (reproduced on both v1.8.4 and v1.9.0)

The same VMI spec with only the hook sidecar removed starts successfully with 2 GPUs and the original memory config. The same VMI spec with the hook kept but reduced to 1 GPU also starts successfully. Only the combination of (hook present) AND (2+ VFIO passthrough devices) fails.

What you expected to happen

VMs combining a nested-virt vIOMMU hook with multiple VFIO passthrough devices should start successfully, the same way they do with either the hook alone (1 device) or multiple devices alone (no hook).

How to reproduce it

1. Bind 2 GPUs to vfio-pci, add both to permittedHostDevices.
2. Deploy an onDefineDomain hook sidecar (via ConfigMap + hooks.kubevirt.io/hookSidecars annotation) that adds to the domain XML's qemu:commandline:

   -machine kernel-irqchip=split
   -device intel-iommu,intremap=on,caching-mode=on,eim=on

(standard pattern for exposing a vIOMMU to the L1 guest so it can do its own VFIO passthrough to a nested L2 guest) 3. Define a VMI with domain.devices.gpus listing 2 devices, the hook annotation from step 2, and normal guest memory (reproduced with both 96Gi/1Gi-hugepages and 8Gi/no-hugepages). 4. Start the VM. 5. compute container starts, QEMU launches, and fails ~4-15 seconds later with the vfio_container_dma_map(...) = -12 error above. virt-launcher tears the domain down cleanly and the pod cycles.

Isolation matrix (all reproduced on the same host/kernel)

Suspected cause

virt-handler calculates the pod's RLIMIT_MEMLOCK ceiling (set via a privileged prlimit() call before QEMU starts) based solely on the VMI's declared gpus/hostDevices count and guest memory — see the fix in #17863 / #17805, which scales this per declared device. That calculation has no visibility into additional QEMU devices injected via a sidecar hook's onDefineDomain — the hook mutates the libvirt domain XML after virt-handler has already set the process's memlock ceiling externally, and hooks are intentionally opaque to KubeVirt's device/resource accounting by design.

An emulated vIOMMU with caching-mode=on requires QEMU to pre-populate and shadow IOMMU translation tables up front (necessary specifically because real hardware devices are also passed through), which increases the amount of memory that needs to be pinned beyond a plain multi-device passthrough setup. With 1 real device this fits inside the existing per-device margin; with 2+ it does not.

This is likely not fixable by having virt-handler "see into" hook output (hooks are intentionally opaque), but may need either: (a) a way for a hook or VMI annotation to request additional memlock headroom, or (b) documented guidance that nested-virt vIOMMU hooks combined with N>1 passthrough devices require external memlock tuning that KubeVirt does not currently support.

Environment

• KubeVirt version: reproduced on v1.8.4 and v1.9.0 (not fixed by the v1.9.0 per-device memlock scaling patch)
• Kubernetes version:
• OS/kernel: reproduced on Ubuntu 24.04 (6.8.0-124-generic); likely version-independent given the mechanism
• Architecture: amd64
• GPUs: NVIDIA AD104GL_L4 (2x)
• Host memory: 756Gi total, 150Gi+ free at time of failure (ruling out real memory pressure)
• Hook sidecar: custom onDefineDomain hook injecting -machine kernel-irqchip=split and -device intel-iommu,intremap=on,caching-mode=on,eim=on for nested virtualization / L1→L2 VFIO passthrough support

Additional context

This is a narrow-impact case for us specifically (3 VMs total use this hook; this is not a fleet-wide GPU passthrough issue), but it blocks the specific architecture of running nested virtualization with multi-GPU passthrough at L1, which we'd guess is a real pattern for anyone building similar L1→L2 GPU passthrough setups.

Related, longer-standing gap: #6824 ("cannot run memory locked vm in kubevirt") reported the same underlying limitation back in 2021 — a hook attempting to set memory.locked/a memlock hard limit with no supported way for KubeVirt to account for it — and was closed stale without a fix landing. This issue is a specific, reproducible instance of that same underlying limitation: hooks have no supported channel to request additional locked-memory headroom from virt-handler's RLIMIT_MEMLOCK calculation.