Hello All- I need to compute the gradient of output of a neural network w.r.t input. In my case output is vector and input is also vector. I was trying to use gradient routine from Flux package but it was complaing that out put is not scaler.I will appreciate if can help me out with this. Thank y…

I guess this may help: [image] Computing function Jacobian from Zygote.pullback Machine Learning Zygote documentation on custom adjoints states dx/dy (in the notation from the docs) is the Jacobian when x, y are vectors. I am struggling in relating this to an implementati…

Hello! It’s difficult to diagnose the issue here without a MWE, can you read through the instructions in Please read: make it easier to help you first?

Thanks! Sure I wrote following routine function net_f(x, t, w, b; act=σ) u_nn = net_u(x, t, w, b; act=σ) # u_nn is of size (25600, 1) ## x, t, w and b are initialized as parameters Tracker.back!(u_nn, ones(size(x)[1])) u_x = Tracker.grad(x) u_t = Tracker.grad(t) return u_x,…

This seems like you are using Flux with a Tracker.jl ad. Is this intentional? The current version of Flux uses Zygote AD, which has a different syntax. Which version of Flux do you use?

@Tomas_Pevny Yes I am using Tracker.jl and using Flux.jl. I can update that to latest version. Thanks!

Any particular reason for using 0.9.0?

@Tomas_Pevny Any pointers towards my question. Will really appreciate. Thanks!

As the error complains. The input for the gradient must be a scalar function (like a L2 loss eg.). Did you have a look at the recent Flux documentation? I believe it’s well explained there. And as mentioned above, without a MWE it’s really hard to help you :roll_eyes:

Autograd in Flux

General Usage

asyrov March 3, 2021, 1:50am 2

I guess this may help:

Topic		Replies	Views
How to use gradient of neural network as the loss function? Machine Learning question	13	2842	March 23, 2021
Gradient of gradient Machine Learning	9	1337	November 6, 2020
Flux loss: Gradient wrt input leads to empty gradient wrt parameters or to "can't differentiate foreigncall" Machine Learning flux , forwarddiff , diffeqflux	3	605	April 8, 2022
Gradient error in Flux model inputs Machine Learning question , flux , zygote	5	1367	January 13, 2021
Compute gradients in neuralODE with Zygote Machine Learning	3	289	August 24, 2023

Autograd in Flux

Related topics