#4301·horovod

Will Horovod support the symmetric memory feature of NCCL 2.27?

Author: jywangxCreated Aug 12, 2025Updated Aug 12, 2025
Labelsenhancement

Is your feature request related to a problem? Please describe. In a single-node scenario, the symmetric memory feature of NCCL 2.27 can provide significant performance improvements for Allreduce, Allgather, and Reduce-Scatter operations, accelerating both model training and inference.

https://developer.nvidia.com/blog/enabling-fast-inference-and-resilient-training-with-nccl-2-27/

Describe the solution you'd like Choose whether to use symmetric memory-accelerated collective communication operations at runtime via environment variables.