torchcomms: a modern PyTorch communications API
torchcomms is a new experimental communications API for PyTorch. This provides both the high level collectives API as well as several out of the box backends.
torchcomms requires the following software and hardware:
torchcomms is available on PyPI and can be installed using pip. Alternatively, you can build torchcomms from source.
You can install torchcomms and PyTorch (2.11+) using pip:
# Cuda 12.6
pip install torch torchcomms --index-url https://download.pytorch.org/whl/cu126
# Cuda 12.8
pip install torch torchcomms --index-url https://download.pytorch.org/whl/cu128
# Cuda 13.0
pip install torch torchcomms --index-url https://download.pytorch.org/whl/cu130You can install torchcomms and PyTorch nightly builds using pip:
# Cuda 12.6
pip install --pre torch torchcomms --index-url https://download.pytorch.org/whl/nightly/cu126
# Cuda 13.0
pip install --pre torch torchcomms --index-url https://download.pytorch.org/whl/nightly/cu130
# Cuda 13.2
pip install --pre torch torchcomms --index-url https://download.pytorch.org/whl/nightly/cu132Alternatively, you can build torchcomms from source. If you want to build the NCCLX backend, we recommend building it under a virtual conda environment. Run the following commands to build and install torchcomms:
# Create a conda environment
conda create -n torchcomms python=3.10
conda activate torchcomms
# Clone the repository
git clone [email protected]:meta-pytorch/torchcomms.git
cd torchcommsNo build needed - uses the library provided by PyTorch
If you want to install the third-party dependencies directly from conda, run the following command:
USE_SYSTEM_LIBS=1 ./build_ncclx.shIf you want to build and install the third-party dependencies from source, run the following command:
./build_ncclx.shInstall some prerequisites
conda install conda-forge::glog=0.4.0 conda-forge::gflags conda-forge::fmt -yEnvironment variables to find rocm/rccl headers
export ROCM_HOME=/opt/rocm
export RCCL_INCLUDE=$ROCM_HOME/include/rccl./build_rccl.shInstall some prerequisites
conda install conda-forge::glog=0.4.0 conda-forge::gflags conda-forge::fmt -yEnvironment variables to find rocm/rcclx headers
export BUILD_DIR=${PWD}/comms/rcclx/develop/build/release/build
export ROCM_HOME=/opt/rocm
export RCCLX_INCLUDE=${BUILD_DIR}/include/rccl
export RCCLX_LIB=${BUILD_DIR}/lib./build_rcclx.shTIP: Default builds both gfx942 and gfx950 and can take 1hr+. Narrow to your GPU:
MI300X/MI325X (gfx942):
./build_rcclx.sh --amdgpu_targets gfx942MI350X/MI355X (gfx950):
./build_rcclx.sh --amdgpu_targets gfx950Detect your arch if unsure:
rocminfo | grep -m1 gfxSource Intel oneAPI environment (update path to your oneAPI installation)
export INTEL_ONEAPI=/path/to/intel/oneapi # e.g., /opt/intel/oneapi or ~/intel/oneapi
source $INTEL_ONEAPI/compiler/latest/env/vars.sh
source $INTEL_ONEAPI/ccl/latest/env/vars.shEnable XCCL backend and install
export USE_XCCL=ON
export USE_NCCL=OFF
export USE_NCCLX=OFF
export USE_TRANSPORT=OFF
pip install --no-build-isolation -v .Set backend env vars before installing. For RCCLX-only:
export USE_NCCL=OFF
export USE_NCCLX=OFF
export USE_GLOO=OFF
export USE_RCCL=OFF
export USE_RCCLX=ON(See Build Configuration below for defaults and other mixes.)
# Install PyTorch (if not already installed)
pip install -r requirements.txt
pip install --no-build-isolation -v .
# Note: For installing torchcomms with RCCL or RCCLX backend, make sure to turn off the other backends:
USE_NCCL=OFF USE_NCCLX=OFF USE_GLOO=OFF USE_RCCL=OFF USE_RCCLX=ON USE_TRANSPORT=OFF pip install --no-build-isolation -v .You can customize the build by setting environment variables before running pip install:
# Enable/disable specific backends (ON/OFF or 1/0)
export USE_NCCL=ON # Default: ON
export USE_NCCLX=ON # Default: ON
export USE_GLOO=ON # Default: ON
export USE_RCCL=OFF # Default: OFF
export USE_RCCLX=OFF # Default: OFF
export USE_XCCL=OFF # Default: OFFThen run:
# Install PyTorch (if not already installed)
pip install -r requirements.txt
pip install --no-build-isolation -v .Here's a simple example demonstrating synchronous AllReduce communication across multiple GPUs:
…To run this example with multiple processes (one per GPU):
# Using torchrun (recommended)
torchrun --nproc_per_node=2 example.py
# Or using python -m torch.distributed.launch
python -m torch.distributed.launch --nproc_per_node=2 example.pyTo run this example with multiple nodes:
torchrun --nnodes=2 --nproc_per_node=8 --node_rank=0 --rdzv-endpoint=":" example.pytorchrun --nnodes=2 --nproc_per_node=8 --node_rank=1 --rdzv-endpoint=":" example.pyIn the example above, we perform the following steps:
new_comm() creates a communicator with the specified backendtorchcomms also supports asynchronous operations for better performance.
Here is the same example as above, but with asynchronous AllReduce:
…See the CONTRIBUTING file for how to help out.
Source code is made available under a BSD 3 license, however you may have other legal obligations that govern your use of other content linked in this repository, such as the license or terms of service for third-party data and models.
torchcomms backends include third-party source code may be using other licenses. Please check the directory and relevant files to verify the license.
For convenience some of them are listed below:
No open issues yet, or sync has not completed.