# Autograd through loss function with derivatives in

**URL:** <https://discourse.julialang.org/t/autograd-through-loss-function-with-derivatives-in/57864>\
**Category:** Machine Learning\
**Created:** [March 24, 2021, 12:09pm UTC](https://discourse.julialang.org/t/autograd-through-loss-function-with-derivatives-in/57864 "2021-03-24T12:09:45Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![RynoLaubscher](https://avatars.discourse-cdn.com/v4/letter/r/8baadc/32.png) [@RynoLaubscher](https://discourse.julialang.org/u/RynoLaubscher)\
**Post date:** [March 25, 2021, 6:51am UTC](https://discourse.julialang.org/t/autograd-through-loss-function-with-derivatives-in/57864/2 "2021-03-25T06:51:46Z")

</div>

Looks like using ReverseDiff over Zygote works, see this discussion [post](https://discourse.julialang.org/t/how-to-use-gradient-of-neural-network-as-the-loss-function/50569/14) for the discussion. Here is my loss function with using Zygote to get the gradients:

```julia
layer(W, b, x) = tanh.(W * x .+ b);
function loss(W, b, x)
    u(x) = layer(W, b, x)[1,:]
    v(x) = layer(W, b, x)[2,:]
    ∂u∂x = Zygote.gradient(a -> sum(u(a)), x)[1][1,:]
    ∂v∂x = Zygote.gradient(a -> sum(v(a)), x)[1][1,:]
    mean(∂u∂x.^2) + mean(∂v∂x.^2) #just a simple loss function, not physical
end

```

Here is the gradient of the loss function with respect to the model weights and biases:

```julia
const W, b = rand(2,2), rand(2);
const input = (W, b);
const diff = ReverseDiff.compile(ReverseDiff.GradientTape((a,b) -> loss(a,b,x0_t0), input));
a = ReverseDiff.gradient!(map(copy, input), diff, input);
a[1]
2×2 Array{Float64,2}:
 0.118311 0.0
 0.126966 0.0

```

---

_[View the full topic](https://discourse.julialang.org/t/autograd-through-loss-function-with-derivatives-in/57864)._