#4485·mlx

[BUG] JACCL segfaults in mx.distributed.init(backend="jaccl") on 2-node Mac-Mac with RDMA devices present

Author: cdvankammenCreated Sep 9, 2026Updated Sep 11, 2026

Description

mx.distributed.init(backend="jaccl") segfaults immediately (signal 11) during 2-node Mac↔Mac initialization, even when both hosts have working RDMA devices (ibv_devices enumerates rdma_en2) and Thunderbolt 40 Gb/s links confirmed. The segfault is 100% reproducible on fresh builds.

This differs from #3777 (closed by #4180) which covered the no-verbs-device case. In our case, RDMA devices ARE present — the crash happens deeper in the initialization path.

The same 2-node setup works fine with MlxRing (socket-based backend).

Environment

  • Hardware: 2× Apple Silicon Macs, 16 GB RAM, connected by direct Thunderbolt cable (40 Gb/s confirmed via system_profiler SPThunderboltDataType)
  • macOS: 26.6.2 (latest)
  • MLX: 0.32.0.dev20260806+cc3f3e60 (from rltakashige/mlx-jaccl-fix-small-recv fork, branch address-rdma-gpu-locks, commit cc3f3e60)
  • Both hosts verified:
    • rdma_ctl statusenabled
    • ibv_devicesrdma_en2 present (real verbs devices)
    • Thunderbolt link: active, 40 Gb/s

Steps to Reproduce

  1. Connect two Apple Silicon Macs via Thunderbolt cable
  2. Enable RDMA on both hosts (rdma_ctl enable, cold boot if needed)
  3. Verify ibv_devices shows rdma_en2 on both hosts
  4. Run mx.distributed.init(backend="jaccl") with both hosts in the group

Expected Behavior

JACCL initialization completes, allowing distributed operations over Thunderbolt RDMA.

Actual Behavior

Segfault immediately during initialization:

Fatal Python error: Segmentation fault
signal=11 (SIGSEGV)

Stack trace points into libjaccl.dylib — the shared-buffer / protection-domain registration path during mx.distributed.init(backend="jaccl").

Isolation Evidence

Test Result
2-node MlxJaccl (RDMA), min_nodes: 2 Segfault signal 11 on both ranks
2-node MlxRing (socket), min_nodes: 2 Works — round-trip inference confirmed
Single-node on each Mac Works

The crash is isolated to JACCL initialization over RDMA. The Ring backend uses the same topology and works. This isolates the bug to the JACCL/RDMA initialization path specifically.

Related

  • Continuation of discussion in #3777 (comment by @cdvankammen, 2026-08-07) — maintainer @erwinzhang7 confirmed our failure is distinct from the #4180 fix
  • MlxRing (non-RDMA socket backend) works as a workaround