theano.gradient.verify_grad lack of checking the validity of inputs
Author: cheyenneeCreated Dec 4, 2023Updated Dec 19, 2024
problem:
theano.gradient.verify_grad doesnt check the validity of inputs. In following snippet code, input_1 and input_alpha should have same shape. When I input them into theano.tensor.nnet.relu, this API throws error. While in theano.gradient.verify_grad, it outputs None without error.
repo code:
import theano
import theano.tensor as T
import numpy as np
def custom_activation(x, alpha):
return theano.tensor.nnet.relu(x, alpha)
x1 = T.tensor3('x1')
alpha = T.tensor4('alpha')
output = custom_activation(x1, alpha)
loss = T.sum(output ** 2)
grad_x1, grad_alpha = T.grad(loss, [x1, alpha])
input_1 = np.random.random((1, 2, 3)).astype('float32')
input_alpha = np.random.random((1, 2, 1, 1)).astype('float32')
rng = np.random.RandomState(123)
print(theano.gradient.verify_grad(custom_activation, pt=[input_1, input_alpha], rng=rng))Source: Theano/Theano