# Jacobian of NN: Mutating arrays is not supported

**URL:** <https://discourse.julialang.org/t/jacobian-of-nn-mutating-arrays-is-not-supported/55053>\
**Category:** Machine Learning\
**Created:** [February 11, 2021, 10:42am UTC](https://discourse.julialang.org/t/jacobian-of-nn-mutating-arrays-is-not-supported/55053 "2021-02-11T10:42:24Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sunny](https://avatars.discourse-cdn.com/v4/letter/s/7c8e57/32.png) [@Sunny](https://discourse.julialang.org/u/Sunny)\
**Post date:** [February 11, 2021, 10:42am UTC](https://discourse.julialang.org/t/jacobian-of-nn-mutating-arrays-is-not-supported/55053/1 "2021-02-11T10:42:24Z")

</div>

I’m trying to take derivative of the jacobian of neural network. But there emerges mutating arrays problem

```julia
nn = Chain(Dense(2,10, tanh), Dense(10,1))
println(Flux.params(nn)) 
jac₁ = x -> ForwardDiff.jacobian(nn, x) # with respect to x
jac₁(rand(2))
grads = Zygote.gradient(()->jac₁(rand(2))[1], params(nn))
grads.grads

```

Is there any way to overcome this in julia?

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [February 11, 2021, 12:47pm UTC](https://discourse.julialang.org/t/jacobian-of-nn-mutating-arrays-is-not-supported/55053/2 "2021-02-11T12:47:57Z")

</div>

I’m not sure this is exactly what you want, but I think things may work better with ForwardDiff over Zygote, the opposite order to what you had. Something like this:

```julia
julia> grads = x -> Zygote.gradient(() -> only(nn(x)), params(nn))
#35 (generic function with 1 method)

julia> grads(rand(2))[params(nn)[1]]
10×2 Matrix{Float64}:
  0.46342 0.249064
 -0.232392 -0.124899
...

julia> ForwardDiff.jacobian(x -> grads(x)[params(nn)[1]], rand(2))
20×2 Matrix{Float64}:
  0.715101 -0.00995561
 -0.00685393 0.132286
...

# next parameter, you could build up IdDict like Grads() if desired
julia> ForwardDiff.jacobian(x -> grads(x)[params(nn)[2]], rand(2))  
10×2 Matrix{Float64}:
 -0.00455624 0.00412416
  0.0606107 -0.00800904
...

```

Note that Zygote#master has a `jacobian` function, which will also work with implicit Params. But ForwardDiff is always going to require explicit input. They way this is done above is, I guess, pretty inefficient, since it re-calculates the whole `grads(x)` for every parameter.

---

<div class="post-metadata">

**Author:** ![Sunny](https://avatars.discourse-cdn.com/v4/letter/s/7c8e57/32.png) [@Sunny](https://discourse.julialang.org/u/Sunny)\
**Post date:** [February 11, 2021, 2:20pm UTC](https://discourse.julialang.org/t/jacobian-of-nn-mutating-arrays-is-not-supported/55053/3 "2021-02-11T14:20:08Z")

</div>

> [@mcabbott](#):
>
> `ForwardDiff.jacobian(x -> grads(x)[params(nn)[2]], rand(2))`

This is so weird. You just changed the order of differentiation and it works. Thanks