tf.math.log_sigmoid loses a finite second-derivative signal for float64 input
Issue type
Bug
Have you reproduced the bug with TensorFlow Nightly?
Yes
Source
source
TensorFlow version
tf 2.21.0
Custom code
Yes
OS platform and distribution
Linux Ubuntu 22.04
Mobile device
No response
Python version
Python 3.13
Bazel version
No response
GCC/compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current behavior?
tf.math.log_sigmoid returns zero for a second-derivative component whose correctly rounded float64 reference value is finite, normal, and nonzero.
For log_sigmoid(x), the second derivative is -sigmoid(x) * sigmoid(-x). At x = -37.42994775023705, the high-precision reference rounded to float64 is approximately -5.551115123125775e-17.
However, TensorFlow autodiff returns -0.0 for this second derivative. The forward value is finite and correctly represented; the discrepancy is in the second-order autodiff result.
Expected behavior?
The second derivative of tf.math.log_sigmoid at x = -37.42994775023705 should be close to -5.551115123125775e-17, not -0.0.
Standalone code to reproduce the issue
import os
os.environ.setdefault("CUDA_VISIBLE_DEVICES", "-1")
os.environ.setdefault("TF_CPP_MIN_LOG_LEVEL", "3")
import tensorflow as tf
x_value = -37.42994775023705
expected = -5.551115123125775e-17
rtol = 1e-6
x = tf.Variable(x_value, dtype=tf.float64)
with tf.GradientTape() as outer:
with tf.GradientTape() as inner:
y = tf.math.log_sigmoid(x)
g1 = inner.gradient(y, x)
g2 = outer.gradient(g1, x)
actual = float(g2)
rel_err = abs(actual - expected) / abs(expected)
print(f"x: {x_value!r}")
print(f"forward value: {float(y)!r}")
print(f"first derivative: {float(g1)!r}")
print(f"tensorflow second derivative: {actual!r}")
print(f"expected second derivative: {expected!r}")
print(f"relative error: {rel_err:.3e}")
if rel_err > rtol:
print("BUG REPRODUCED: tf.math.log_sigmoid loses a finite second-derivative signal")
else:
print("not reproduced")Relevant log output
x: -37.42994775023705
forward value: -37.42994775023705
tensorflow second derivative: -0.0
expected second derivative: -5.551115123125775e-17
relative error: 1.000e+00
BUG REPRODUCED: tf.math.log_sigmoid loses a finite second-derivative signalSource: tensorflow/tensorflow