[bug]: Recording hijacks a same-pod sidecar's port, breaking the app's egress TLS/mTLS to it

Is there an existing issue for this?

• I have searched and didn't find similar issue

Current behavior

When an application and a TLS dependency (including mTLS) run as two containers in the same Kubernetes pod, starting a recording breaks every TLS call the app makes to that dependency. The calls fail for as long as the recording is active and succeed again as soon as it stops.

Scope, established by the endpoint matrix in the comment below: plaintext egress to the same sidecar is unaffected, and the same TLS call to a sidecar in a different pod works. The trigger is a TLS handshake against a port keploy relocated inside the same pod; mTLS is one case of it, not the cause.

The cause is keploy's ingress capture. The agent hooks bind() via eBPF (ListenForIngressEvents → WatchBindEvents, pkg/agent/proxy/incoming/incoming.go), relocates every bound port in the pod, and takes the original port for itself:

Started ingress forwarding {"orig_port": 8443, "new_port": 38235}   <- the sidecar
Started ingress forwarding {"orig_port": 8080, "new_port": 35441}   <- the app

netstat inside the pod during recording — nginx is no longer on 8443:

Proto Local Address   State    PID/Program name
tcp   0.0.0.0:8443    LISTEN   -                     <- keploy
tcp   0.0.0.0:38235   LISTEN   96/nginx: master pr   <- the real sidecar, relocated
tcp   0.0.0.0:8080    LISTEN   -                     <- keploy
tcp   :::35441        LISTEN   67/java               <- the real app, relocated

Because both containers share one network namespace, the app's egress to https://localhost:8443 now lands on keploy's own ingress listener instead of the sidecar. The egress path then fails to dial what it believes is the dependency:

ERROR failed to dial the conn to destination server
  {"Destination Address": "[127.0.0.1]:8443",
   "server address": "localhost:8443",
   "error": "tls: first record does not look like a TLS handshake"}
ERROR failed to handle the client connection
  {"error": "tls: first record does not look like a TLS handshake"}

There is no configuration that avoids this: port capture is driven by bind() interception, so every sidecar in the pod has its port taken. Only TLS listeners are broken by it — a plaintext listener keeps working despite also being relocated.

Expected

Recording should not break the application. Either exclude a port from ingress capture when it is also an egress destination inside the same pod, or resolve the app's egress to the relocated port rather than to keploy's own listener.

Knock-on effects

1. The app receives no diagnosable error. The connection is closed with no TLS alert and no response — the client sees only EOF. The reason appears solely in the agent log.

2. The failed call is recorded as expected behavior. The test case captures the error response produced by the broken dependency, so auto-replay then reports it as a failure (Total tests: 2, Passed: 1, Failed: 1, Some testsets failed due to mock differences). The recorded test-set is wrong rather than merely incomplete.

3. The diagnostic hint misleads. The error suggests an address-family problem:

   "next_step": "check which address FAMILY the dependency binds — a container published port is IPv4-only by default..."

   The dependency binds correctly; keploy relocated it. This sends anyone debugging in the field toward IPv4/IPv6 instead of the port rewrite.

Steps to Replicate

1. Deploy one pod with two containers:
   • java-app (main) — on GET /call, makes one outbound HTTPS request to https://localhost:8443/api/data presenting a client certificate; GET /health makes no outbound call.
   • mtls-sidecar — nginx listening on 8443 with ssl_verify_client on.
2. With no recording active, GET /call returns 200 and nginx reports $ssl_client_verify = SUCCESS.
3. Start a recording on this deployment.
4. GET /call now fails on every attempt. GET /health keeps returning 200.
5. kubectl exec <pod> -c mtls-sidecar -- netstat -lntp — nginx is on a relocated high port and keploy holds 8443.
6. Stop the recording; GET /call returns 200 again.

Minimal manifests (self-signed CA generated with openssl; no external dependencies):

# nginx.conf (ConfigMap) — the mTLS sidecar
events { worker_connections 1024; }
error_log /dev/stderr info;
http {
  access_log /dev/stdout combined;
  server {
    listen 8443 ssl;
    ssl_certificate     /certs/server.crt;
    ssl_certificate_key /certs/server.key;
    ssl_client_certificate /certs/ca.crt;
    ssl_verify_client on;                 # this makes it mTLS
    location /api/data {
      default_type application/json;
      return 200 '{"ok":true,"client_dn":"$ssl_client_s_dn","verify":"$ssl_client_verify"}';
    }
  }
}

spec:
  containers:
    - name: java-app                      # MAIN
      image: <app>
      ports: [{ containerPort: 8080 }]
      env:
        - { name: MTLS_URL, value: "https://localhost:8443/api/data" }
      volumeMounts: [{ name: certs, mountPath: /certs, readOnly: true }]
    - name: mtls-sidecar                  # SIDECAR
      image: nginx:1.27-alpine
      ports: [{ containerPort: 8443 }]
      volumeMounts:
        - { name: certs, mountPath: /certs, readOnly: true }
        - { name: nginx-conf, mountPath: /etc/nginx/nginx.conf, subPath: nginx.conf, readOnly: true }

Logs (if any)

All four views, same recording, same seconds:

# keploy agent
ERROR failed to dial the conn to destination server
  {"Destination Address": "[127.0.0.1]:8443",
   "error": "tls: first record does not look like a TLS handshake"}

# application
[app] upstream FAILED in 147ms: java.io.IOException: HTTP/1.1 header parser received no bytes
[app]   caused by: java.io.EOFException: EOF reached while reading

# nginx sidecar — never completes a handshake
[info] *3 client closed connection while SSL handshaking, client: 127.0.0.1, server: 0.0.0.0:8443

# k8s-proxy — the failure is then recorded as the expected result
Total tests: 2   Total test passed: 1   Total test failed: 1
Some testsets failed due to mock differences.

Operating system

Linux

System Info (uname -a)

Linux <host> 5.10.0-39-amd64 #1 SMP Debian 5.10.251-1 (2026-03-09) x86_64 GNU/Linux

OS Release Info

PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
VERSION_ID="11"

Docker Info

• Docker version 29.4.1
• kind v0.31.0 (single-node cluster)
• Agent runtime image: keploy/enterprise:v3.8.27

Your Environment

• Kubernetes via kind, single node
• Recording started from the Keploy dashboard (k8s-proxy sidecar-injection mode, daemonset.enabled=false)
• Both containers in one pod, therefore one network namespace — this is the precondition for the bug

Version

v3.8.27 (agent image keploy/enterprise:v3.8.27)

Secondary issue found in the same investigation

With no client key configured, the same setup fails earlier and just as silently. applyMTLSClientCert (pkg/agent/proxy/proxy.go) returns an error when the key is empty, and the caller drops the connection:

ERROR failed to apply mTLS client cert
  {"error": "failed to read private key from outgoing options"}
ERROR failed to handle the client connection

The application again sees only an EOF. Since keploy cannot obtain the client's private key from a handshake, an app doing egress mTLS without a configured key is unsupported by design — but it should say so (fail the recording, or warn), rather than silently killing every such connection at runtime.