Conv2D 默认初始化（`Normal(0, √(2/fan_in))`）在没有归一化深层/共享权重结构的情况下，会导致激活指数放大，从而引起训练发散和 NaN 导数。
作者: youge325创建于 2026年8月27日更新于 2026年8月27日
- Actual comparison:
torch.nn.Conv2d(16,16,3,padding=1)default weight std=0.0475(max 0.083);paddle.nn.Conv2Dsame shape std=0.1164(max 0.412). - Real scenario: We encountered in a dynamic depth VSR network with shared weights (about 46-69 times of the same batch convolution module being called in a pyramid layer-wise manner): on GPU, the L1 loss in the second step exploded from 0.0006 to 255; on CPU, backward propagation produced NaN gradients for 28/44 parameters. After explicitly changing all convolutions to
U(±1/√fan_in), the training curves of the two identical implementations by paddle and torch almost completely overlapped (150 steps total loss …
内容来源: PaddlePaddle/Paddle