Lack of Input Validation in theano.gradient.verify_grad

Issue with theano.gradient.verify_grad Failing to Check Input Validity The theano.gradient.verify_grad function does not validate the compatibility of input shapes for the tested function. In the example below, input_1 and input_alpha are of mismatched shapes, but theano.gradient.verify_grad fails to detect the issue and outputs None without any error. However, directly using these inputs with theano.tensor.nnet.relu causes an error.

Problematic Code `import theano import theano.tensor as T import numpy as np

Custom activation function using Theano's ReLU

def custom_activation(x, alpha): return theano.tensor.nnet.relu(x, alpha)

Define symbolic tensors

x1 = T.tensor3('x1') alpha = T.tensor4('alpha')

Compute output and gradients

output = custom_activation(x1, alpha) loss = T.sum(output ** 2) grad_x1, grad_alpha = T.grad(loss, [x1, alpha])

Define inputs with mismatched shapes

input_1 = np.random.random((1, 2, 3)).astype('float32') # Shape: (1, 2, 3) input_alpha = np.random.random((1, 2, 1, 1)).astype('float32') # Shape: (1, 2, 1, 1)

Verify gradients

rng = np.random.RandomState(123) print(theano.gradient.verify_grad(custom_activation, pt=[input_1, input_alpha], rng=rng)) `

Explanation of the Issue Shape Mismatch: The inputs input_1 and input_alpha must have compatible shapes for broadcasting when passed to theano.tensor.nnet.relu. In this case, they are not compatible. No Error from verify_grad: Despite the incompatibility, theano.gradient.verify_grad does not validate the input shapes before proceeding, leading to an incorrect None output instead of raising an error. Error in theano.tensor.nnet.relu: If custom_activation is used directly, the underlying theano.tensor.nnet.relu function correctly raises an error due to the shape mismatch.