使用 DDP 时的后向问题
作者: UESTC-LXD创建于 2024年1月9日更新于 2024年7月26日
在使用 DDP 时遇到问题。我应该如何在该位置找到警告? 变量._execution_engine.run_backward( # 通过 C++ 引擎运行反向传递 /home/ps/anaconda3/envs/py38/lib/python3.8/site-packages/torch/autograd/init.py:197: UserWarning: Grad strides do not match bucket view strides. This may indicate grad was not created according to the gradient layout contract, or that the param's strides changed since DDP was constructed. This is not an error, but may impair performance. grad.sizes() = [32, 64, 1, 1, 1], strides() = [64, 1, 64, 64, 64] bucket_view.sizes() = [32, 64, 1, 1, 1], strides() = [64, 1, 1, 1, 1] (在 ../torch/csrc/distributed/c10d/reducer.cpp:325 处内部触发。)
内容来源: lucidrains/imagen-pytorch