I am trying to display gradients of weights and biases in each layer of a multi layer network as below:
f = rand(2,10) y = rand(10) dims = [2, 4, 2, 1] layers = [Dense(dims[i], dims[i+1], Flux.relu) for i in 1:length(dims)-1]; m = Chain(layers...) L(x, y) = Flux.Losses.mse(m(x), y) gs = gradient(() -> L(f, y), ps)
I got like below:
But how to see the gradient of each layer?
For a single layer, I could see gradients as gs[W]… But this is not working multilayer… How to solve this?