[BUG] JACCL segfaults in mx.distributed.init(backend="jaccl") on 2-node Mac-Mac with RDMA devices present
Description
mx.distributed.init(backend="jaccl") segfaults immediately (signal 11) during
2-node Mac↔Mac initialization, even when both hosts have working RDMA devices
(ibv_devices enumerates rdma_en2) and Thunderbolt 40 Gb/s links confirmed.
The segfault is 100% reproducible on fresh builds.
This differs from #3777 (closed by #4180) which covered the no-verbs-device case. In our case, RDMA devices ARE present — the crash happens deeper in the initialization path.
The same 2-node setup works fine with MlxRing (socket-based backend).
Environment
- Hardware: 2× Apple Silicon Macs, 16 GB RAM, connected by direct Thunderbolt cable (40 Gb/s confirmed via
system_profiler SPThunderboltDataType) - macOS: 26.6.2 (latest)
- MLX:
0.32.0.dev20260806+cc3f3e60(fromrltakashige/mlx-jaccl-fix-small-recvfork, branchaddress-rdma-gpu-locks, commitcc3f3e60) - Both hosts verified:
rdma_ctl status→enabledibv_devices→rdma_en2present (real verbs devices)- Thunderbolt link: active, 40 Gb/s
Steps to Reproduce
- Connect two Apple Silicon Macs via Thunderbolt cable
- Enable RDMA on both hosts (
rdma_ctl enable, cold boot if needed) - Verify
ibv_devicesshowsrdma_en2on both hosts - Run
mx.distributed.init(backend="jaccl")with both hosts in the group
Expected Behavior
JACCL initialization completes, allowing distributed operations over Thunderbolt RDMA.
Actual Behavior
Segfault immediately during initialization:
Fatal Python error: Segmentation fault
signal=11 (SIGSEGV)
Stack trace points into libjaccl.dylib — the shared-buffer / protection-domain
registration path during mx.distributed.init(backend="jaccl").
Isolation Evidence
| Test | Result |
|---|---|
2-node MlxJaccl (RDMA), min_nodes: 2 |
Segfault signal 11 on both ranks |
2-node MlxRing (socket), min_nodes: 2 |
Works — round-trip inference confirmed |
| Single-node on each Mac | Works |
The crash is isolated to JACCL initialization over RDMA. The Ring backend uses the same topology and works. This isolates the bug to the JACCL/RDMA initialization path specifically.
Related
- Continuation of discussion in #3777 (comment by @cdvankammen, 2026-08-07) — maintainer @erwinzhang7 confirmed our failure is distinct from the #4180 fix
- MlxRing (non-RDMA socket backend) works as a workaround
Source: ml-explore/mlx