Will Horovod support the symmetric memory feature of NCCL 2.27?
Author: jywangxCreated Aug 12, 2025Updated Aug 12, 2025
Labelsenhancement
Is your feature request related to a problem? Please describe. In a single-node scenario, the symmetric memory feature of NCCL 2.27 can provide significant performance improvements for Allreduce, Allgather, and Reduce-Scatter operations, accelerating both model training and inference.
https://developer.nvidia.com/blog/enabling-fast-inference-and-resilient-training-with-nccl-2-27/
Describe the solution you'd like Choose whether to use symmetric memory-accelerated collective communication operations at runtime via environment variables.
Source: horovod/horovod