paddle.nn.functional.silu backward returns -Inf for finite float64 gradient at x=-709
Author: ALinrunrunCreated Aug 6, 2026Updated Aug 6, 2026
Labelsstatus/new-issuetype/bug-report
bug描述 Describe the Bug
paddle.nn.functional.silu has an incorrect backward result for float64 input x = -709.0.
For SiLU, the function is:
silu(x) = x * sigmoid(x)Its derivative is:
silu'(x) = sigmoid(x) + x * sigmoid(x) * (1 - sigmoid(x))At x = -709.0, the correct derivative is approximately:
-8.614807714413836e-306This is a finite, normal float64 value. However, Paddle autograd returns -Inf.
Environment
- PaddlePaddle version: 3.3.1
- NumPy version: 2.2.6
- dtype:
float64 - Device: CPU
Minimal reproducible code
import paddle
import paddle.nn.functional as F
paddle.set_device("cpu")
x = paddle.to_tensor(
-709.0,
dtype="float64",
stop_gradient=False,
)
y = F.silu(x)
y.backward()
wrong = float(x.grad)
expected = -8.614807714413836e-306
print("wrong:", wrong)
print("expected:", expected)Actual behavior
wrong: -infExpected behavior
expected: -8.614807714413836e-306paddle.nn.functional.silu backward should return a finite float64 gradient for x = -709.0.
Instead, Paddle returns -Inf, even though the correct gradient is finite and representable in float64.
其他补充信息 Additional Supplementary Information
No response
Source: PaddlePaddle/Paddle