L2: nodeAssigned event is emitted on every reconcile, not only when the announcing node changes
MetalLB Version: v0.15.3 Deployment method: Charts Main CNI: Calico v3.31.4 (VXLAN) Kubernetes Version: 1.34.5 Cluster Distribution: kubeadm
Describe the bug
The nodeAssigned event is emitted on every reconcile, not only when the announcing node changes.
In handleService, the event is placed outside the if !c.announced[protocol][name] check which is computed just above it (speaker/main.go#L498-L512).
So if some external controller rewrites a service's endpoints on a timer, every write causes a reconcile, and every reconcile emits the event once more. The node never changes, but the event keeps repeating.
On our cluster (v0.15.3, L2 mode), five LoadBalancer services with externally managed endpoints gave 1,101,899 occurrences of this event, around 6 per minute each, for weeks together. All the other LoadBalancer services on the same cluster were between 130 and 190. There was no failover, no speaker restart and no L2Advertisement change during this period.
To Reproduce
- Set up L2 with one IPAddressPool and one L2Advertisement.
- Create a LoadBalancer service without any selector.
- Create an EndpointSlice for it manually, pointing to any reachable pod IP.
- Rewrite that EndpointSlice every 10 seconds, keeping the contents the same. (Patroni does exactly this for the PostgreSQL leader endpoint, every
loop_wait.) - Run
kubectl get events --field-selector reason=nodeAssigned -w.
The count keeps increasing at the same rate as the endpoint writes, with the same message and the same node.
Expected Behaviour
The event should be emitted when the announcing node is assigned or changes. Continuing on the same node is not a change.
Additional Context
Reproducible on main, the code path is unchanged there.
Fix in #3134.
Source: metallb/metallb