Support L2 failover using the Kubernetes out-of-service taint without requiring memberlist

Is your feature request related to a problem?

Is your feature request related to a problem?

When using MetalLB in native L2 mode with memberlist disabled, there is currently no apparent mechanism for MetalLB to recognize that a speaker's Kubernetes node has become unavailable due to a hardware failure and should no longer participate in L2 ownership.

This is particularly problematic when using Kubernetes' node.kubernetes.io/out-of-service=nodeshutdown:NoExecute workflow for hardware failure recovery.

For example, in a 3-node cluster:

Node1 -> announces VIP A Node2 -> announces VIP B Node3 -> available as another potential speaker

If Node1 experiences a hardware failure, Kubernetes marks it NotReady. Applying:

kubectl taint nodes <node-name> node.kubernetes.io/out-of-service=nodeshutdown:NoExecute (see https://kubernetes.io/docs/concepts/cluster-administration/node-shutdown/#non-graceful-node-shutdown for more info on this taint)

causes the MetalLB speaker pod on Node1 to be removed, while also allowing other workloads, including StatefulSets, PVC-backed workloads, and VMs, to recover on the remaining nodes.

However, when memberlist is disabled, MetalLB does not appear to use the Kubernetes node's NotReady state or node.kubernetes.io/out-of-service taint to determine that Node1 is no longer a viable L2 announcer.

As a result, VIP A is not transferred to another eligible speaker. The existing ServiceL2Status for VIP A disappears rather than being recreated for another node.

This is different from memberlist-enabled operation, where the speakers use Hashicorp/memberlist gossip to detect that the failed speaker is no longer a member and remove it from consideration.

The problem is therefore specifically relevant to deployments that disable memberlist and rely on Kubernetes state for node failure handling.

Describe the solution you'd like

I'd like MetalLB L2 mode to be able to recognize that a Kubernetes node which has become unavailable due to a hardware failure is no longer eligible to announce L2 VIPs, even when memberlist is disabled.

When a node becomes NotReady and is explicitly marked with:

node.kubernetes.io/out-of-service=nodeshutdown:NoExecute

I would expect MetalLB to stop considering that node for L2 ownership and allow another eligible speaker to assume ownership of any VIPs previously announced by that node.

The desired behavior is:

Before failure:

Node1 -> VIP A Node2 -> VIP B

Node1 hardware failure:

Node1 -> NotReady + out-of-service -> speaker removed

After failure:

Node2/Node3 -> VIP A Node2 -> VIP B

The goal is to integrate MetalLB L2 failover with the Kubernetes node failure/out-of-service workflow rather than requiring memberlist to be enabled solely for failure detection.

There are legitimate reasons for disabling memberlist. For example, MetalLB may be deployed as part of a Kubernetes cluster running on an edge or IoT system where CPU, memory, and other system resources are limited. In such environments, avoiding the additional resource consumption and overhead of memberlist is desirable while still requiring reliable L2 failover based on Kubernetes node state.

Possible approaches could include honoring the node.kubernetes.io/out-of-service taint, considering Kubernetes node readiness, or another mechanism the maintainers feel is more appropriate.

Additional context

MetalLB version tested:

v0.15.3

Cluster:

k3s 3 nodes flannel CNI

MetalLB is deployed using a Helm chart.

Memberlist is intentionally disabled:

speaker: memberlist: enabled: false

Disabling memberlist is a deliberate configuration choice rather than a workaround for this issue. In resource-constrained edge/IoT deployments, reducing the resource footprint of cluster components can be important, and memberlist may be disabled to avoid the additional resource consumption associated with maintaining the gossip cluster.

All Kubernetes nodes are intended to be eligible L2 announcers; there are no nodeSelectors on the L2Advertisement resources.

The environment contains multiple L2 advertisements/IP pools. During the failure scenario, an advertisement whose VIP was already owned by a surviving node continues to work. Only the VIP whose announcing node experienced the hardware failure stops working.

I also verified that the behavior is not specific to the node selectors or FRR configuration:

Removing nodeSelectors does not change the behavior. Disabling FRR does not change the behavior. Restarting the MetalLB controller and surviving speakers does not restore the VIP. The surviving speaker continues to successfully announce the other VIP. The failed node's speaker pod is removed after applying the out-of-service taint. The MetalLB controller has permission to list and watch Nodes.

I reviewed the L2 speaker eligibility logic in speaker/layer2_controller.go, specifically speakersForAds(). When memberlist is enabled, non-members are removed from consideration using memberlist membership information. When memberlist is disabled, that failure-detection mechanism is unavailable.

The remaining logic appears to check conditions such as NodeNetworkUnavailable and the node.kubernetes.io/exclude-from-external-load-balancers label, but does not appear to use NotReady or the node.kubernetes.io/out-of-service taint to exclude an unavailable node.

The Kubernetes out-of-service taint is specifically intended to identify nodes that are no longer serving workloads, particularly in node failure scenarios, and enables recovery of resources such as volumes and workloads from a failed node. In this environment it is already being used successfully for that purpose. (please see https://kubernetes.io/docs/concepts/cluster-administration/node-shutdown/#non-graceful-node-shutdown for more info on this taint)

I'd be happy to submit a PR implementing this behavior but I would prefer to get guidance from the maintainers on the desired semantics first, particularly whether MetalLB should honor the out-of-service taint, node readiness, or another mechanism when determining L2 announcer eligibility with memberlist disabled.

I've read and agree with the following

• I've checked all open and closed issues and my request is not there.
• I've checked all open and closed pull requests and my request is not there.