Direct path periodically drops to relay every ~50-75s behind CGNAT with absolute UDP mapping timeout (T-Mobile) — make-before-break path rotation?
Author: huangyingwCreated Jul 17, 2026Updated Sep 3, 2026
LabelsNAT Traversal
Setup
- Server: Raspberry Pi on residential fiber (public IPv4, symmetric NAT router, no UPnP), iroh 1.0.2 / noq 1.0.1, fixed UDP port,
Builder::external_addr+ pkarr publishing with an unfiltered-ish addr filter (so clients do get direct candidates). - Client: iOS (iroh-ffi 1.1.0) on T-Mobile US cellular (CGNAT, 172.56.0.0/12 pool).
- Holepunching succeeds: connections reach
SELECTED directwithin ~1s of establishment. (This is the same deployment as my comment on #4134 — direct now works after we started publishing candidates.)
Observed: direct path drops back to relay every ~50–75s, then re-punches
Server-side Connection::path_events() log for 4 consecutive sessions (timestamps relative):
SELECTED direct → +50s closed → SELECTED relay → +1s SELECTED direct (re-punched)
SELECTED direct → +48s closed → SELECTED relay → +26s SELECTED direct
SELECTED direct → +57s closed → SELECTED relay → +4s SELECTED direct
SELECTED direct → +75s closed → SELECTED relay → (conn ended)The pattern repeats for the lifetime of every connection: during continuous media streaming this means a relay (rate-limited) window roughly once per minute.
Analysis
- Per-path 5s heartbeats (
HEARTBEAT_INTERVAL) are present, so this is not an idle-traffic problem. - The timing strongly matches CGNAT absolute UDP mapping timeout / port rebinding (T-Mobile is commonly reported at ~30–60s, regardless of traffic). Once the mapping is re-bound, the old 4-tuple is a black hole; iroh correctly detects it after
PATH_MAX_IDLE_TIMEOUT(15s), falls back to relay, and re-punches (5s attempt interval) — i.e. detection+rebuild is working as designed, worst-case ~26s on relay. - Key point: with an absolute timeout, keep-alives cannot save the mapping, and while the mapping is still alive the same internal source port keeps mapping to the same external tuple — so "refreshing" over the same socket can't produce a fresh mapping either. Surviving this NAT class seems to require make-before-break with a tuple change: proactively establish a new direct path (new local source port via rebind, or a different remote port candidate) shortly before the old mapping's expected death, switch over, and let the old path drain.
Questions / proposals
- Is there an intended mechanism for this scenario in the current multipath design (noq)? If not, would you be open to:
a. Proactive path rotation: when a direct path's age approaches an (adaptive or configurable) threshold, start a holepunch round for a fresh tuple while the old path is still selected — zero relay window on switch; or at least
b. Faster rebuild knobs:
PATH_MAX_IDLE_TIMEOUT/HOLEPUNCH_ATTEMPTS_INTERVAL(and friends) arepub(crate)consts today; making them configurable would let deployments on high-churn NATs shrink the relay window (15s+ → a few s) without protocol changes. - We're happy to contribute either (and can validate against this real T-Mobile CGNAT deployment with server-side path-event data). Which direction fits your roadmap?
(Server-side per-path close stats — rtt/tx/rx/lost/blackhole counters — are being collected on our side and can be attached if useful.)
Source: n0-computer/iroh