replicateServices.fromHost fails for LoadBalancer Services because status is copied to a headless ClusterIP Service
What happened?
When networking.replicateServices.fromHost maps a host Service of type
LoadBalancer into a virtual cluster, vCluster creates the target Service as a
headless ClusterIP Service but then attempts to copy the source Service's
status.loadBalancer into it.
Kubernetes rejects that update:
Service "<service>" is invalid: status.loadBalancer.ingress: Forbidden:
may only be used when spec.type is 'LoadBalancer'The reconciliation then returns from the failed status update before creating the target Endpoints. The mapped Service therefore has no endpoints and is not reachable from workloads in the virtual cluster.
vCluster version
Observed with vCluster 0.36.0.
The same control flow is still present on current main as of commit
39c7f42dd6d756bfa1b8c4cb224a9f1a554a3dff.
How can we reproduce it?
Create a host-cluster
LoadBalancerService and wait for it to receive a non-emptystatus.loadBalancer.ingress.Configure vCluster to replicate it:
networking:
replicateServices:
fromHost:
- from: example/source-service
to: example/source-service- Inspect the mapped Service inside the virtual cluster:
$ kubectl -n example get service source-service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
source-service ClusterIP None <none> 80/TCP- Inspect the vCluster syncer logs:
Service "source-service" is invalid: status.loadBalancer.ingress: Forbidden:
may only be used when spec.type is 'LoadBalancer'- Verify that no Endpoints were created:
$ kubectl -n example get endpoints source-service
Error from server (NotFound): endpoints "source-service" not foundRoot cause
In pkg/controllers/servicesync/servicesync.go,
syncServiceAndEndpoints creates the target Service with:
Spec: corev1.ServiceSpec{
Ports: fromService.Spec.Ports,
ClusterIP: corev1.ClusterIPNone,
},Because Spec.Type is omitted, the target Service is a ClusterIP Service.
On the next reconciliation, the controller checks only the source Service type before copying its load-balancer status:
if fromService.Spec.Type == corev1.ServiceTypeLoadBalancer &&
!apiequality.Semantic.DeepEqual(
fromService.Status.LoadBalancer,
toService.Status.LoadBalancer,
) {
toService.Status.LoadBalancer = fromService.Status.LoadBalancer
return ctrl.Result{}, e.To.GetClient().Status().Update(ctx, toService)
}The target is not a LoadBalancer, so Kubernetes rejects the status update.
Since the controller returns at that point, it never reaches the subsequent
Endpoints creation logic.
What did you expect to happen?
The mapped Service and its Endpoints should be created successfully, regardless
of whether the source Service is a ClusterIP or LoadBalancer Service.
For the existing headless-Service implementation, the target does not need the source load-balancer status: traffic is routed using the Endpoints generated from the source Service's ClusterIP.
Suggested fix
Only synchronize status.loadBalancer when the target Service is also of type
LoadBalancer, or skip load-balancer status synchronization for
fromHost mappings that use CreateEndpoints.
For example:
if fromService.Spec.Type == corev1.ServiceTypeLoadBalancer &&
toService.Spec.Type == corev1.ServiceTypeLoadBalancer &&
!apiequality.Semantic.DeepEqual(
fromService.Status.LoadBalancer,
toService.Status.LoadBalancer,
) {
// Update status.
}A regression test should reconcile a host LoadBalancer Service with a
non-empty load-balancer status and verify that:
- reconciliation succeeds;
- the virtual Service remains valid;
- no invalid load-balancer status is copied; and
- the virtual Endpoints are created from the host Service ClusterIP.
Related issues
- #1926 reports that host-to-vCluster mappings create a headless target Service.
- #2717 proposes changing how the target ClusterIP is set.
Those are related to the target Service representation, but do not prevent the
invalid status.loadBalancer update described here.
Source: loft-sh/vcluster