I was wondering if it was possible to parametrize matrices such that zygote’s gradient would only be performed on the selected coefficients of the matrix.
Some dummy code would look like:
theta = randn(2) W = [theta theta; 0.0 theta] b = randn(100) x = randn(2, 100) gs = gradient(w -> sum(w*x+b), W)
However the gradient would only be performed on the values of theta, keeping W[2, 1] set to 0.0 for the entire gradient descent. Also this would imply that the gradients for theta and theta be equal at all times.
Am I completely going at this wrong way? or is it just not possible in julia?