Hi Everyone, I would like to know how I can manually calculate for example the mean over an array of gradients of a neural network. I have the following setup : In the training loop I calculate the gradient of the loss grad = Flux.gradient((m) -> abs2(y[i] - m(x[i])), net)[1] at the end I want…

Average of Zygote gradients

mcabbott May 4, 2023, 12:11pm 3

It sounds like you want Optimisers.AccumGrad

2 Likes