Helm 4 orders the validating webhooks before user CRs, so a first install that includes IPAddressPool fails (and never converges under a retrying installer)

MetalLB Version

0.15.3 (also verified the 0.16.1 chart is affected — see below)

Deployment method

Charts

Main CNI

flannel (k3s default)

Kubernetes Version

v1.36.4

Cluster Distribution

k3s

Describe the bug

Installing the MetalLB chart in the same Helm pass as an IPAddressPool/L2Advertisement fails on a fresh install under Helm 4:

Error: INSTALLATION FAILED: Internal error occurred: failed calling webhook
"ipaddresspoolvalidationwebhook.metallb.io": failed to call webhook:
Post "https://metallb-webhook-service.metallb-system.svc:443/validate-metallb-io-v1beta1-ipaddresspool?timeout=10s":
no endpoints available for service "metallb-webhook-service"

The chart itself ships no CRs, so this only shows up in the umbrella-chart pattern recommended in #2241 — wrap the MetalLB chart as a dependency and put your IPAddressPool / L2Advertisement in the parent's templates/. That is a common layout for ArgoCD/Flux and for anything installing in one pass.

Why it is new. Helm sorts resources by kind, and unknown kinds sort differently between majors. Under Helm 3 the CRs happened to be applied before the ValidatingWebhookConfigurations, so they were admitted while no webhook was registered. Under Helm 4 the webhooks are applied first, so the CRs are now admitted through a webhook whose backing controller Deployment was created moments earlier in the same pass and is not ready. With crds.validationFailurePolicy: Fail (the default) that is a hard rejection.

Same chart, same cluster, same values, apply-only:

It does not self-heal under a retrying installer. k3s's helm-controller (klipper-helm, which is Helm 4.1.4) runs helm uninstall before each retry — which deletes the controller Deployment backing the webhook — then reinstalls and applies the CRs while the replacement pod is still starting. Each attempt destroys the prerequisite the next one needs, so it loops indefinitely (CrashLoopBackOff, 6+ restarts observed, LoadBalancer stuck <pending>). Any installer that retries by reinstalling will behave the same way.

To Reproduce

Reproduced in two independent environments — a VM appliance and a plain rancher/k3s:v1.36.4-k3s1 container under Docker — so it is not specific to a CNI or network setup.

1. Pre-install the MetalLB CRDs (we place them as cluster-scoped manifests; crds.enabled=false in the chart).
2. Build an umbrella chart with metallb as a dependency and an IPAddressPool + L2Advertisement in templates/, per #2241.
3. helm install <name> ./umbrella -n metallb-system --create-namespace --set metallb.enabled=true using Helm 4.

The install fails with the error above. The same command under Helm 3 succeeds.

Expected Behavior

A first install that includes CRs should either succeed, or fail in a way that converges on retry.

Four mitigations measured against a real cluster:

Worth noting the first row: --server-side=false is the obvious-looking fix and changes nothing.

Ignore seems the reasonable answer, since it does not disable validation — it only changes what the apiserver does when it cannot reach the validator, and in steady state the webhook is up and validates as before.

So the ask is small: would you consider either

1. documenting this caveat (today crds.validationFailurePolicy has an empty description in the chart README and nothing in NOTES.txt, so there is no hint that Fail makes a first install fragile), and/or
2. revisiting whether Fail is the right default now that Helm 4 orders webhooks ahead of CRs?

Happy to send a PR for either if you have a preference on the shape.

Additional Context

• Chart 0.16.1 is affected identically: its templates/webhooks.yaml is byte-identical to 0.15.3's, validationFailurePolicy: Fail is still the default, and there are no helm.sh/hook annotations anywhere in the templates. So this is not something a version bump resolves.
• Related but distinct: #1597 (webhook failures from caBundle/networking causes) and #2241 (the umbrella-chart recommendation this report follows).
• We are working around it downstream with crds.validationFailurePolicy: Ignore; filing because the recommended pattern now breaks by default under Helm 4, and klipper-helm shipping Helm 4 means every k3s cluster is on that path.