[RFC] IPv6 / dual-stack support (TCP control plane + RDMA data plane)

Motivation

Some RDMA fabrics are IPv6-only: high-bandwidth NICs get ULA + SLAAC IPv6 addresses and no IPv4 at all. 3FS currently cannot run on such fabrics — Address is IPv4-only by design, so neither the TCP control plane nor the RDMA data plane can bind or dial IPv6 endpoints.

Root cause

src/common/utils/Address.h:

• uint32_t ip + static_assert(sizeof(Address) == sizeof(uint64_t));
• operator uint64_t() is used as the hash/equality/serialization primitive;
• is_serde_copyable makes RPC serialization a raw 8-byte memcpy.

Fitting a 16-byte IPv6 address requires reworking this whole chain, not a one-line patch.

What we have implemented and validated

We have a working dual-stack implementation (~19 files, +292/−87 against current main), running for several weeks on a 2-node test deployment:

• Address: 8B → 20B (16-byte address + family), IPv6 literal parsing (TCP://[fd00::1]:8000), FNV hash + explicit operator==, factory helpers.
• IfAddrs: collect AF_INET6 addresses (skipping link-local).
• Listener: new ipv6_only option (default false, IPv4 deployments unaffected); ConnectionPool / TcpSocket generalized via sockaddr_storage with per-family branches.
• RDMA: rdma_resolve_addr with sockaddr_in6, dual-stack peer info in QP metadata exchange, RoCE v2 IPv6 GID selection.
• Unit tests updated/added (TestAddress IPv6 cases, TestEcho, TestIBSocket).

Validation: full cluster (mgmtd / meta / storage / FUSE) over IPv6 for both TCP control plane and RDMA data plane; real LLM checkpoint loads through FUSE at multi-GB/s per NIC; unit tests pass.

The design question we want maintainer input on

We kept is_serde_copyable and simply memcpy 20 bytes. This is the smallest change, but it makes the wire format incompatible with existing v4 binaries: old and new nodes cannot coexist, so no rolling upgrade. Options we see:

1. Accept as a breaking change (flag day / major version bump) — simplest.
2. Move Address to field-based serde for cross-version compatibility — cleaner long-term, touches RPC schema.
3. Version-negotiated dual format — most compatible, most complexity.

Which direction would you prefer? We are happy to adapt the implementation.

Proposed contribution plan

If there is interest, we would split the work into reviewable PRs:

1. Address dual-stack core + unit tests;
2. TCP path (IfAddrs / Listener / ConnectionPool / TcpSocket);
3. RDMA path (IBConnect / IBSocket / GID selection).

Happy to share more benchmark details or rebase onto any preferred design.