#2309 [Bug] MLX Ring fails on 2-node Thunderbolt cluster — master and worker pick different IPs for the master (Ethernet vs Thunderbolt) · exo

Summary

When running a 2-node EXO cluster connected via Thunderbolt, the MLX Ring backend fails to connect because the master and the worker write the master node's IP into the MLX hostfile using two different addresses:

• Master (rank 0) uses the Ethernet IP (192.168.204.30, interface en0)                                                                                  
• Worker (rank 1) uses the Thunderbolt bridge IP (10.10.0.2, interface bridge0)

The worker tries to connect to 10.10.0.2:PORT, but the master only listens on 192.168.204.30:PORT → Connection refused (error 61) → ring init fails with [ring] Couldn't connect (error: 60).

Affected version

• EXO v1.0.71 (macOS DMG)

Environment

• 2 × Mac mini M4 base (16 GB), macOS 26.x
• One Thunderbolt 4 cable between the machines (forms a Thunderbolt Bridge, e.g. 10.10.0.2 / 10.10.0.3)
• Both machines also on the same Ethernet LAN (192.168.204.0/24)
• Node A runs as --force-master; node B runs as worker with --bootstrap-peers /ip4/10.10.0.2/tcp/50420/p2p/<master_id>                                  
• Both nodes run with --offline

Reproduce

1. Connect two Mac minis via Thunderbolt (a Thunderbolt Bridge service appears, e.g. 10.10.0.2 and 10.10.0.3).
2. Also connect both to the same Ethernet network (192.168.204.x).

Actual

Master (rank 0):                                                                                                                                        
                                                                                                                                                        
  rank 0 hostfile: ... hosts: ["192.168.204.30:63889", "192.168.204.155:63889"]                                                                         
                                                                                                                                                
Worker (rank 1):                                                       

  rank 1 hostfile: ... hosts: ["10.10.0.2:63889", "192.168.204.155:63889"]                                                                              
                                                                                                                                                        
Worker connects to 10.10.0.2:63889, but master only listens on 192.168.204.30:63889:

  [ring] Rank 1 connecting to 0                                                
  [ring] Attempt 0 waiting 1000 ms (error: 61)
  ...           
  RuntimeError: [ring] Couldn't connect (error: 60)

Root-cause notes

Interface classification is correct on the master (networksetup -listallhardwareports maps bridge0 → thunderbolt, en0 → ethernet), yet master still picks the Ethernet IP while the worker picks the TB IP.

The topology is asymmetric between the two nodes:

• worker → master: rdma_en2 + SocketConnections (10.10.0.2, 192.168.204.30, 169.254.114.34)                                                             
• master → worker: only rdma_en2 (no SocketConnection)

This suggests _find_connection_ip(node_i, node_j) (which only reads the directed source→sink edges) yields different candidate IPs depending on which node is the caller, so each node ends up choosing a different IP for the same target.

Also, if only TB is present (Ethernet IP removed from the master), the master picks a link-local 169.254.x.x rather than the TB IP — still no match.

Reproduces consistently across restarts on v1.0.71.

Request

Please review _find_connection_ip / _find_ip_prioritised so both nodes resolve to the same IP for the master in the ring hostfile, consistently preferring Thunderbolt.