tf.nn.selu returns wrong second derivative near zero for negative float64 input

Issue type

Bug

Have you reproduced the bug with TensorFlow Nightly?

Yes

Source

source

TensorFlow version

tf 2.21.0

Custom code

Yes

OS platform and distribution

Linux Ubuntu 22.04

Mobile device

No response

Python version

Python 3.11

Bazel version

No response

GCC/compiler version

No response

CUDA/cuDNN version

No response

GPU model and memory

No response

Current behavior?

tf.nn.selu returns an incorrect second derivative for a small negative float64 input.

For SELU, the negative branch is a scaled ELU branch. Around x < 0, the second derivative should be finite and nonzero. At x = -1e-300, the high-precision analytic reference is approximately 1.7580993408473768.

However, TensorFlow autodiff returns 0.0 for the second derivative at this input. Peer autodiff backends such as JAX and PyTorch compute the expected finite value, so this appears to be a higher-order backward-pass issue rather than a float64 limitation.

Expected behavior?

The second derivative of tf.nn.selu at x = -1e-300 should be close to 1.7580993408473768, not 0.0.

Standalone code to reproduce the issue

import os
os.environ.setdefault("CUDA_VISIBLE_DEVICES", "-1")
os.environ.setdefault("TF_CPP_MIN_LOG_LEVEL", "3")

import tensorflow as tf

x = -1e-300
truth = 1.7580993408473768

v = tf.Variable(x, dtype=tf.float64)

with tf.GradientTape() as tape2:
    with tf.GradientTape() as tape1:
        y = tf.nn.selu(v)
    g1 = tape1.gradient(y, v)
g2 = tape2.gradient(g1, v)

actual = float(g2)
rel_err = abs(actual - truth) / abs(truth)

print(f"x: {x!r}")
print(f"first derivative: {float(g1)!r}")
print(f"second derivative: {actual!r}")
print(f"expected second derivative: {truth!r}")
print(f"relative error: {rel_err:.3e}")

if rel_err > 1e-6:
    print("BUG REPRODUCED: tf.nn.selu second derivative is wrong near zero for negative float64 input")
else:
    print("not reproduced")

Relevant log output

tf: d^2 selu(-1e-300) = 0.0  truth=1.7580993408473768  rel_err=1.000e+00  WRONG