Cilium fails to properly write eBPF maps in some circumstances

Is there an existing issue for this?

• I have searched the existing issues

Version

equal or higher than v1.18.8 and lower than v1.19.0

What happened?

Cilium fails to properly write eBPF maps in some circumstances, seemingly related to Intel ICE drivers(?). This issue has been observed to chronically affect bare metal servers with intel 810 series NICs, but strangely, never any other type of host. It's not known whether ICE plays a factor here, or this is some other strange co-incidence. But this pattern has been observed 1000+ machines for at least a year.

How can we reproduce the issue?

This issue has been reproduced on Cilium 1.18.7, and 1.19.2. on Ubuntu 20.04, and Ubuntu 22.04, with kernels 5.15.0-72 and 6.8.0-101

Install Cilium 1.19.2 on Ubuntu 22.04, with a 6.8.0-101-generic kernel on a bare-metal server with Intel 810 series nic using the latest Intel ICE driver 2.3.15. This does not appear to affect bare metal servers running i40e driver NICs.

modinfo ice filename: /lib/modules/6.8.0-101-generic/updates/drivers/net/ethernet/intel/ice/ice.ko firmware: updates/intel/ice/ddp/ice.pkg version: 2.3.15

Over time, you will observe eventually pods begin to fail all layer 3 communication. You will be able to inspect the cilium ip cache bpf map as improperly empty. This will be in contrast to what the API believes should be in the ebpf map. Cilium will log that it is appropriate regenerating the ipcache map frequently, but this is not correct.

time=2026-03-23T20:48:00.47345889Z level=debug source=/go/src/github.com/cilium/cilium/pkg/k8s/watchers/pod.go:246 msg="Updated ipcache map entry on pod add" module=agent.controlplane.k8s-watcher k8sPodName=bpinske-debugging-churn-29571648-0-6c6vp k8sNamespace=pool podIP="" podIPs=[] hostIP=""

I am investigating further the exact contents of the BPF_MAP_UPDATE_ELEM syscall with strace to try and understand why or how this is being blackholed.

Attached is some cilium debug information. I am specifically including the output of cilium bpf endpoint list and cilium endpoint list demonstrating the discprepancy and incorrectness of the bpf map state.

cilium-dbg-bpf-endpoint-list.md cilium-dbg-endpoint-list.md

Cilium Version

1.18.7, 1.19.2

Kernel Version

5.15.0-72-generic and 6.8.0-101-generic

Kubernetes Version

1.34.4

Regression

No response

Sysdump

cilium-debuginfo-20260406-184840.459+0000-UTC.md

Relevant log output

Anything else?

No response

Cilium Users Document

• Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

• I agree to follow this project's Code of Conduct