Local-registry deploy deadlocks during `docker pull` of large image layers (net-ssh reverse forward)

Summary

When using the built-in local registry (registry: { server: localhost:5555 }), kamal deploy hangs in Kamal::Cli::Build#pull as soon as the image has a layer larger than ~27 MiB. The reverse SSH tunnel that exposes the local registry to the host freezes mid-transfer and the deploy never proceeds.

Environment

• Kamal 2.12.0 (the relevant code is unchanged on main)
• Deploy machine: macOS (Apple Silicon, Homebrew Ruby), net-ssh 7.3.2
• Single amd64 target host
• registry.server: localhost:5555 (local registry), proxy: false

Symptom

Build and push app image...            # ok
Setting up local registry port forwarding to <host>...
  INFO Running docker image rm --force localhost:5555/app:<sha> on <host>
  INFO Running docker pull localhost:5555/app:<sha> on <host>
# …hangs here forever; the kamal process sits in I/O wait at 0% CPU

On the host the forwarded connection stalls mid-pull — ss -tni shows the byte counter frozen at a fixed offset with TCP retransmits. In my case it stalled deterministically at bytes_received=28665002 (~27.3 MiB), reproduced at the same offset on every attempt. Small images (changed layer < ~27 MiB) pull fine, which is why this often only appears once an app grows.

Root cause

Kamal::Cli::Build::PortForwarding sets up the reverse tunnel directly with net-ssh:

Net::SSH.start(host, ssh_options[:user], **ssh_options.except(:user)) do |ssh|
  ssh.forward.remote(port, "localhost", port, "127.0.0.1") { ... }
  ssh.loop(0.1) { @done ? (ssh.forward.cancel_remote(port, "127.0.0.1"); break) : true }
end

During docker pull the bulk data flows host → deploy-machine (net-ssh is the receiver on the forwarded channel). That channel is pumped by the single ssh.loop(0.1) running in a background thread, while the main thread is simultaneously driving the pull command over its own net-ssh/SSHKit connection. Under sustained inflow the forwarded channel's receive pump starves (local-window exhaustion / GVL contention between the two in-process net-ssh loops), the host stops receiving window updates, and the transfer wedges.

This is a net-ssh transport problem, not the network — a plain OS ssh reverse tunnel carries the identical pull with no issues:

ssh -N -R 127.0.0.1:5555:localhost:5555 -o ExitOnForwardFailure=yes <host>
# then `docker pull localhost:5555/app:<sha>` on the host completes normally

Likely related

• #1690 — same area; surfaces as Errno::ERANGE: Result too large - send(2) from net-ssh during the forwarded pull on macOS.
• #1674 / #1676 — fixed a readiness race (confirm ports are up before pulling) but not the bulk-transfer stall.
• #1587 — "kamal deploy hangs indefinitely after Pull app image finishes".

Proposed fix

Establish the reverse forward via the OS ssh client inside PortForwarding instead of net-ssh's in-process forwarding (mapping the existing ssh_options to the equivalent ssh flags). This keeps the local-registry design unchanged and is reliable under load. I'll open a PR with this shortly — happy to adjust if you'd prefer a net-ssh-level fix.

Workaround for anyone hitting this now

Carry the tunnel over the OS ssh client. Either reopen Kamal::Cli::Build::PortForwarding to shell out to ssh -R (loaded before kamal runs), or pre-load the image onto the host and deploy with --skip-push. Both unblock it.