Baike.dev
Log in
Back to tool/Back to issues
#48794·cilium

Cilium connectivity tests with DNS egress rules are broken in Gardener Kubernetes clusters

Author: psheichelbergCreated Sep 17, 2026Updated Sep 17, 2026
Labelskind/bugneeds/triagekind/community-report

Is there an existing issue for this?

  • I have searched the existing issues

Version

equal or higher than v1.20.1 and lower than v1.21.0

What happened?

I ran the Cilium connectivity tests (cilium connectivity test) against a Gardener Kubernetes cluster. Some of the tests are failing because they use CiliumNetworkPolicies with DNS egress rules to the ports 53 (and 5353 for OpenShift) but the Gardener DNS is listening on port 8053 (see Gardener docs).

These are the failing tests:

  • client-egress-l7-named-port
  • client-egress-l7-port-range
  • client-egress-l7
  • client-egress-tls-sni-denied
  • client-egress-tls-sni
  • to-entities-world-port-range
  • to-entities-world
  • to-fqdns-with-proxy
  • to-fqdns

And these are CiliumNetworkPolicies used during those tests:

Since they already include rules specific to OpenShift and nodelocaldns I would like to rules specific to Gardener clusters.

How can we reproduce the issue?

  1. Have a Gardener shoot cluster with Cilium networking. (Not sure how to reproduce it without that. I tried to setup Gardener locally to provide the reproducible steps but I could not get it running. Maybe a KinD cluster with a re-configured CoreDNS listening on port 8053 instead of 53 and Cilium would work?)
  2. Run cilium connectivity test against it:
    bash
    cilium connectivity test \
  --test=client-egress-l7-named-port \
  --test=client-egress-l7-port-range \
  --test=client-egress-l7 \
  --test=client-egress-tls-sni-denied \
  --test=client-egress-tls-sni \
  --test=to-entities-world-port-range \
  --test=to-entities-world \
  --test=to-fqdns-with-proxy \
  --test=to-fqdns

Cilium Version

cilium-cli: v0.20.0 compiled with go1.27.0 on linux/amd64
cilium image (default): v1.20.1
cilium image (stable): v1.20.2
cilium image (running): unknown. Unable to obtain cilium version. Reason: release: not found

Running Cilium version according to cilium connectivity test

Cilium version: 1.17.14

Kernel Version

Linux [local-machine] 6.17.0-1032-oem #32-Ubuntu SMP PREEMPT_DYNAMIC Fri Jul 24 05:46:26 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux`
Linux [cluster-node-1] 6.8.0-138-generic #138-Ubuntu SMP PREEMPT_DYNAMIC Fri Jul 31 22:41:49 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Linux [cluster-node-2] 6.12.102-flatcar #1 SMP PREEMPT_DYNAMIC Tue Aug 11 23:10:58 -00 2026 x86_64 AMD EPYC-Milan Processor AuthenticAMD GNU/Linux

Kubernetes Version

Client Version: v1.37.0
Kustomize Version: v5.8.1
Server Version: v1.35.8

Regression

No response

Sysdump

No response

Relevant log output

bash
ℹ️  Monitor aggregation detected, will skip some flow validation steps
ℹ️  Skipping tests that require a node Without Cilium
[…]
 Enabling Hubble telescope...
⚠️  Unable to contact Hubble Relay, disabling Hubble telescope and flow validation: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:4245: connect: connection refused"
ℹ️  Expose Relay locally with:
   cilium hubble enable
   cilium hubble port-forward&
ℹ️  Cilium version: 1.17.14
[cilium-test-1] Running 137 tests ...
[skipped tests …]
[=] [cilium-test-1] Test [to-entities-world] [32/137]
.
  ℹ️  curl stdout:
  :0 -> :0 = 000
  ℹ️  curl stderr:
  curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2000 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
  
  [.] Action [to-entities-world/pod-to-world:https-to-one.one.one.one.-ipv4-0: cilium-test-1/client-7b7877766f-mtrx2 (10.96.2.30) -> one.one.one.one.-https (one.one.one.one.:443)]
  ℹ️   Applying CiliumNetworkPolicy 'client-egress-to-entities-world' to namespace 'cilium-test-1' on cluster kubecontext..
  [-] Scenario [to-entities-world/pod-to-world]
  [.] Action [to-entities-world/pod-to-world:http-to-one.one.one.one.-ipv4-0: cilium-test-1/client-7b7877766f-mtrx2 (10.96.2.30) -> one.one.one.one.-http (one.one.one.one.:80)]
  ❌ command "curl --silent --fail --show-error --connect-timeout 2 --max-time 10 -4 -H Host: one.one.one.one -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code}\n --output /dev/null --retry 3 --retry-all-errors --retry-delay 3 http://one.one.one.one.:80" failed: command failed (pod=cilium-test-1/client-7b7877766f-mtrx2, container=client): command terminated with exit code 28
.  [.] Action [to-entities-world/pod-to-world:https-to-one.one.one.one.-index-ipv4-0: cilium-test-1/client-7b7877766f-mtrx2 (10.96.2.30) -> one.one.one.one.-https-index (one.one.one.one.:443)]
.  [.] Action [to-entities-world/pod-to-world:http-to-one.one.one.one.-ipv4-1: cilium-test-1/client2-d56fbd75d-vvg4j (10.96.2.41) -> one.one.one.one.-http (one.one.one.one.:80)]
.  ❌ command "curl --silent --fail --show-error --connect-timeout 2 --max-time 10 -4 -H Host: one.one.one.one -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code}\n --output /dev/null --retry 3 --retry-all-errors --retry-delay 3 http://one.one.one.one.:80" failed: command failed (pod=cilium-test-1/client2-d56fbd75d-vvg4j, container=client2): command terminated with exit code 28
  ℹ️  curl stdout:
  :0 -> :0 = 000
  ℹ️  curl stderr:
  curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
  
  [.] Action [to-entities-world/pod-to-world:https-to-one.one.one.one.-ipv4-1: cilium-test-1/client2-d56fbd75d-vvg4j (10.96.2.41) -> one.one.one.one.-https (one.one.one.one.:443)]
.  [.] Action [to-entities-world/pod-to-world:https-to-one.one.one.one.-index-ipv4-1: cilium-test-1/client2-d56fbd75d-vvg4j (10.96.2.41) -> one.one.one.one.-https-index (one.one.one.one.:443)]
.  [.] Action [to-entities-world/pod-to-world:http-to-one.one.one.one.-ipv4-2: cilium-test-1/client3-74887fc475-vzmt4 (10.96.0.226) -> one.one.one.one.-http (one.one.one.one.:80)]
.  ❌ command "curl --silent --fail --show-error --connect-timeout 2 --max-time 10 -4 -H Host: one.one.one.one -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code}\n --output /dev/null --retry 3 --retry-all-errors --retry-delay 3 http://one.one.one.one.:80" failed: command failed (pod=cilium-test-1/client3-74887fc475-vzmt4, container=client3): command terminated with exit code 28
  ℹ️  curl stdout:
  :0 -> :0 = 000
  ℹ️  curl stderr:
  curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2000 milliseconds
  
  [.] Action [to-entities-world/pod-to-world:https-to-one.one.one.one.-ipv4-2: cilium-test-1/client3-74887fc475-vzmt4 (10.96.0.226) -> one.one.one.one.-https (one.one.one.one.:443)]
.  [.] Action [to-entities-world/pod-to-world:https-to-one.one.one.one.-index-ipv4-2: cilium-test-1/client3-74887fc475-vzmt4 (10.96.0.226) -> one.one.one.one.-https-index (one.one.one.one.:443)]
.  ℹ️   Deleting CiliumNetworkPolicy 'client-egress-to-entities-world' in namespace 'cilium-test-1' on cluster kubecontext..
[=] [cilium-test-1] Test [to-entities-world-port-range] [33/137]
.
  ℹ️  curl stdout:
  :0 -> :0 = 000
  ℹ️  curl stderr:
  curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
  
  [.] Action [to-entities-world-port-range/pod-to-world:https-to-one.one.one.one.-ipv4-0: cilium-test-1/client3-74887fc475-vzmt4 (10.96.0.226) -> one.one.one.one.-https (one.one.one.one.:443)]
  ℹ️   Applying CiliumNetworkPolicy 'client-egress-to-entities-world-port-range' to namespace 'cilium-test-1' on cluster kubecontext..
  [-] Scenario [to-entities-world-port-range/pod-to-world]
  [.] Action [to-entities-world-port-range/pod-to-world:http-to-one.one.one.one.-ipv4-0: cilium-test-1/client3-74887fc475-vzmt4 (10.96.0.226) -> one.one.one.one.-http (one.one.one.one.:80)]
  ❌ command "curl --silent --fail --show-error --connect-timeout 2 --max-time 10 -4 -H Host: one.one.one.one -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code}\n --output /dev/null --retry 3 --retry-all-errors --retry-delay 3 http://one.one.one.one.:80" failed: command failed (pod=cilium-test-1/client3-74887fc475-vzmt4, container=client3): command terminated with exit code 28
.  [.] Action [to-entities-world-port-range/pod-to-world:https-to-one.one.one.one.-index-ipv4-0: cilium-test-1/client3-74887fc475-vzmt4 (10.96.0.226) -> one.one.one.one.-https-index (one.one.one.one.:443)]
.  [.] Action [to-entities-world-port-range/pod-to-world:http-to-one.one.one.one.-ipv4-1: cilium-test-1/client-7b7877766f-mtrx2 (10.96.2.30) -> one.one.one.one.-http (one.one.one.one.:80)]
.  ❌ command "curl --silent --fail --show-error --connect-timeout 2 --max-time 10 -4 -H Host: one.one.one.one -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code}\n --output /dev/null --retry 3 --retry-all-errors --retry-delay 3 http://one.one.one.one.:80" failed: command failed (pod=cilium-test-1/client-7b7877766f-mtrx2, container=client): command terminated with exit code 28
  ℹ️  curl stdout:
  :0 -> :0 = 000
  ℹ️  curl stderr:
  curl: (28) Resolving timed out after 2000 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2002 milliseconds
curl: (28) Resolving timed out after 2000 milliseconds
  
  [.] Action [to-entities-world-port-range/pod-to-world:https-to-one.one.one.one.-ipv4-1: cilium-test-1/client-7b7877766f-mtrx2 (10.96.2.30) -> one.one.one.one.-https (one.one.one.one.:443)]
.  [.] Action [to-entities-world-port-range/pod-to-world:https-to-one.one.one.one.-index-ipv4-1: cilium-test-1/client-7b7877766f-mtrx2 (10.96.2.30) -> one.one.one.one.-https-index (one.one.one.one.:443)]
.  [.] Action [to-entities-world-port-range/pod-to-world:http-to-one.one.one.one.-ipv4-2: cilium-test-1/client2-d56fbd75d-vvg4j (10.96.2.41) -> one.one.one.one.-http (one.one.one.one.:80)]
.  ❌ command "curl --silent --fail --show-error --connect-timeout 2 --max-time 10 -4 -H Host: one.one.one.one -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code}\n --output /dev/null --retry 3 --retry-all-errors --retry-delay 3 http://one.one.one.one.:80" failed: command failed (pod=cilium-test-1/client2-d56fbd75d-vvg4j, container=client2): command terminated with exit code 28
  ℹ️  curl stdout:
  :0 -> :0 = 000
  ℹ️  curl stderr:
  curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
  
  [.] Action [to-entities-world-port-range/pod-to-world:https-to-one.one.one.one.-ipv4-2: cilium-test-1/client2-d56fbd75d-vvg4j (10.96.2.41) -> one.one.one.one.-https (one.one.one.one.:443)]
.  [.] Action [to-entities-world-port-range/pod-to-world:https-to-one.one.one.one.-index-ipv4-2: cilium-test-1/client2-d56fbd75d-vvg4j (10.96.2.41) -> one.one.one.one.-https-index (one.one.one.one.:443)]
.  ℹ️   Deleting CiliumNetworkPolicy 'client-egress-to-entities-world-port-range' in namespace 'cilium-test-1' on cluster kubecontext..
[skipped tests …]
[=] [cilium-test-1] Test [client-egress-l7-method] [74/137]
..................
[=] [cilium-test-1] Test [client-egress-l7-method-port-range] [75/137]
..................
[=] [cilium-test-1] Test [client-egress-l7] [76/137]
.............
  ℹ️  curl stdout:
  :0 -> :0 = 000
  ℹ️  curl stderr:
  curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
  
  [.] Action [client-egress-l7/pod-to-world:https-to-one.one.one.one.-ipv4-2: cilium-test-1/client2-d56fbd75d-vvg4j (10.96.2.41) -> one.one.one.one.-https (one.one.one.one.:443)]
  ℹ️   Applying CiliumNetworkPolicy 'client-egress-only-dns' to namespace 'cilium-test-1' on cluster kubecontext..
  ℹ️   Applying CiliumNetworkPolicy 'client-egress-l7-http' to namespace 'cilium-test-1' on cluster kubecontext..
  [-] Scenario [client-egress-l7/pod-to-pod]
  [.] Action [client-egress-l7/pod-to-pod:curl-ipv4-0: cilium-test-1/client-7b7877766f-mtrx2 (10.96.2.30) -> cilium-test-1/echo-other-node-5856c44977-d6wm7 (10.96.0.88:8080)]
  [.] Action [client-egress-l7/pod-to-pod:curl-ipv4-1: cilium-test-1/client-7b7877766f-mtrx2 (10.96.2.30) -> cilium-test-1/echo-same-node-6fcd55c86c-9jvnx (10.96.2.100:8080)]
  [.] Action [client-egress-l7/pod-to-pod:curl-ipv4-2: cilium-test-1/client2-d56fbd75d-vvg4j (10.96.2.41) -> cilium-test-1/echo-other-node-5856c44977-d6wm7 (10.96.0.88:8080)]
  [.] Action [client-egress-l7/pod-to-pod:curl-ipv4-3: cilium-test-1/client2-d56fbd75d-vvg4j (10.96.2.41) -> cilium-test-1/echo-same-node-6fcd55c86c-9jvnx (10.96.2.100:8080)]
  [.] Action [client-egress-l7/pod-to-pod:curl-ipv4-4: cilium-test-1/client3-74887fc475-vzmt4 (10.96.0.226) -> cilium-test-1/echo-other-node-5856c44977-d6wm7 (10.96.0.88:8080)]
  [.] Action [client-egress-l7/pod-to-pod:curl-ipv4-5: cilium-test-1/client3-74887fc475-vzmt4 (10.96.0.226) -> cilium-test-1/echo-same-node-6fcd55c86c-9jvnx (10.96.2.100:8080)]
  [-] Scenario [client-egress-l7/pod-to-world]
  [.] Action [client-egress-l7/pod-to-world:http-to-one.one.one.one.-ipv4-0: cilium-test-1/client3-74887fc475-vzmt4 (10.96.0.226) -> one.one.one.one.-http (one.one.one.one.:80)]
  [.] Action [client-egress-l7/pod-to-world:https-to-one.one.one.one.-ipv4-0: cilium-test-1/client3-74887fc475-vzmt4 (10.96.0.226) -> one.one.one.one.-https (one.one.one.one.:443)]
  [.] Action [client-egress-l7/pod-to-world:https-to-one.one.one.one.-index-ipv4-0: cilium-test-1/client3-74887fc475-vzmt4 (10.96.0.226) -> one.one.one.one.-https-index (one.one.one.one.:443)]
  [.] Action [client-egress-l7/pod-to-world:http-to-one.one.one.one.-ipv4-1: cilium-test-1/client-7b7877766f-mtrx2 (10.96.2.30) -> one.one.one.one.-http (one.one.one.one.:80)]
  [.] Action [client-egress-l7/pod-to-world:https-to-one.one.one.one.-ipv4-1: cilium-test-1/client-7b7877766f-mtrx2 (10.96.2.30) -> one.one.one.one.-https (one.one.one.one.:443)]
  [.] Action [client-egress-l7/pod-to-world:https-to-one.one.one.one.-index-ipv4-1: cilium-test-1/client-7b7877766f-mtrx2 (10.96.2.30) -> one.one.one.one.-https-index (one.one.one.one.:443)]
  [.] Action [client-egress-l7/pod-to-world:http-to-one.one.one.one.-ipv4-2: cilium-test-1/client2-d56fbd75d-vvg4j (10.96.2.41) -> one.one.one.one.-http (one.one.one.one.:80)]
  ❌ command "curl --silent --fail --show-error --connect-timeout 2 --max-time 10 -4 -H Host: one.one.one.one -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code}\n --output /dev/null --retry 3 --retry-all-errors --retry-delay 3 http://one.one.one.one.:80" failed: command failed (pod=cilium-test-1/client2-d56fbd75d-vvg4j, container=client2): command terminated with exit code 28
.  [.] Action [client-egress-l7/pod-to-world:https-to-one.one.one.one.-index-ipv4-2: cilium-test-1/client2-d56fbd75d-vvg4j (10.96.2.41) -> one.one.one.one.-https-index (one.one.one.one.:443)]
.  ℹ️   Deleting CiliumNetworkPolicy 'client-egress-only-dns' in namespace 'cilium-test-1' on cluster kubecontext..
  ℹ️   Deleting CiliumNetworkPolicy 'client-egress-l7-http' in namespace 'cilium-test-1' on cluster kubecontext..
[=] [cilium-test-1] Test [client-egress-l7-port-range] [77/137]
..........
  ℹ️  curl stdout:
  :0 -> :0 = 000
  ℹ️  curl stderr:
  curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2000 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
curl: (28) Resolving timed out after 2001 milliseconds
  
  [.] Action [client-egress-l7-port-range/pod-to-world:https-to-one.one.one.one.-ipv4-1: cilium-test-1/client2-d56fbd75d-vvg4j (10.96.2.41) -> one.one.one.one.-https (one.one.one.one.:443)]
  ℹ️   Applying CiliumNetworkPolicy 'client-egress-only-dns' to namespace 'cilium-test-1' on cluster kubecontext..
  ℹ️   Applying CiliumNetworkPolicy 'client-egress-l7-http-port-range' to namespace 'cilium-test-1' on cluster kubecontext..
  [-] Scenario [client-egress-l7-port-range/pod-to-pod]
  [.] Action [client-egress-l7-port-range/pod-to-pod:curl-ipv4-0: cilium-test-1/client2-d56fbd75d-vvg4j (10.96.2.41) -> cilium-test-1/echo-other-node-5856c44977-d6wm7 (10.96.0.88:8080)]
  [.] Action [client-egress-l7-port-range/pod-to-pod:curl-ipv4-1: cilium-test-1/client2-d56fbd75d-vvg4j (10.96.2.41) -> cilium-test-1/echo-same-node-6fcd55c86c-9jvnx (10.96.2.100:8080)]
  [.] Action [client-egress-l7-port-range/pod-to-pod:curl-ipv4-2: cilium-test-1/client3-74887fc475-vzmt4 (10.96.0.226) -> cilium-test-1/echo-other-node-5856c44977-d6wm7 (10.96.0.88:8080)]
  [.] Action [client-egress-l7-port-range/pod-to-pod:curl-ipv4-3: cilium-test-1/client3-74887fc475-vzmt4 (10.96.0.226) -> cilium-test-1/echo-same-node-6fcd55c86c-9jvnx (10.96.2.100:8080)]
  [.] Action [client-egress-l7-port-range/pod-to-pod:curl-ipv4-4: cilium-test-1/client-7b7877766f-mtrx2 (10.96.2.30) -> cilium-test-1/echo-same-node-6fcd55c86c-9jvnx (10.96.2.100:8080)]
  [.] Action [client-egress-l7-port-range/pod-to-pod:curl-ipv4-5: cilium-test-1/client-7b7877766f-mtrx2 (10.96.2.30) -> cilium-test-1/echo-other-node-5856c44977-d6wm7 (10.96.0.88:8080)]
  [-] Scenario [client-egress-l7-port-range/pod-to-world]
  [.] Action [client-egress-l7-port-range/pod-to-world:http-to-one.one.one.one.-ipv4-0: cilium-test-1/client-7b7877766f-mtrx2 (10.96.2.30) -> one.one.one.one.-http (one.one.one.one.:80)]
  [.] Action [client-egress-l7-port-range/pod-to-world:https-to-one.one.one.one.-ipv4-0: cilium-test-1/client-7b7877766f-mtrx2 (10.96.2.30) -> one.one.one.one.-https (one.one.one.one.:443)]
  [.] Action [client-egress-l7-port-range/pod-to-world:https-to-one.one.one.one.-index-ipv4-0: cilium-test-1/client-7b7877766f-mtrx2 (10.96.2.30) -> one.one.one.one.-https-index (one.one.one.one.:443)]
  [.] Action [client-egress-l7-port-range/pod-to-world:http-to-one.one.one.one.-ipv4-1: cilium-test-1/client2-d56fbd75d-vvg4j (10.96.2.41) -> one.one.one.one.-http (one.one.one.one.:80)]
  ❌ command "curl --silent --fail --show-error --connect-timeout 2 --max-time 10 -4 -H Host: one.one.one.one -w %{local_ip}:%{local_port} -> %{remote_ip}:%{remote_port} = %{response_code}\n --output /dev/null --retry 3 --retry-all-errors --retry-delay 3 http://one.one.one.one.:80" failed: command failed (pod=cilium-test-1/client2-d56fbd75d-vvg4j, container=client2): command terminated with exit code 28
.  [.] Action [client-egress-l7-port-range/pod-to-world:https-to-one.one.one.one.-index-ipv4-1: cilium-test-1/client2-d56fbd75d-vvg4j (10.96.2.41) -> one.one.one.one.-https-index (one.one.one.one.:443)]
.  [.] Action [client-egress-l7-port-range/pod-to-world:http-to-one.one.one.one.-ipv4-2: cilium-test-1/client3-74887fc475-vzmt4 (10.96.0.226) -> one.one.one.one.-http

Source: cilium/cilium

View original on GitHubView discussion on GitHub