# Is there an efficient way to compute the Hessian of a NN?

**URL:** <https://discourse.julialang.org/t/is-there-an-efficient-way-to-compute-the-hessian-of-a-nn/26971>\
**Category:** Machine Learning\
**Tags:** flux\
**Created:** [July 30, 2019, 10:08am UTC](https://discourse.julialang.org/t/is-there-an-efficient-way-to-compute-the-hessian-of-a-nn/26971 "2019-07-30T10:08:33Z")\
**Posts on this page:** 1\
**Showing post:** 1

<div class="post-metadata">

**Author:** ![HenriDeh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrideh/32/8316_2.png) [@HenriDeh](https://discourse.julialang.org/u/HenriDeh)\
**Post date:** [July 30, 2019, 10:08am UTC](https://discourse.julialang.org/t/is-there-an-efficient-way-to-compute-the-hessian-of-a-nn/26971/1 "2019-07-30T10:08:33Z")

</div>

`Flux.jacobian(::Chain, ::AbstractArray)` gives the full jacobian of a neural network. `Flux.hessian` however throws the error “output is not scalar”.

Let `m = Chain(Dense(5,3,elu), Dense(3,3))` . What I can do is compute the gradient of each value in the jacobian matrix w.r. to each input using `Flux.gradient(x->Flux.jacobian(m, x)[i], input)` where `i` the index of the jacobian matrix we are interested in differentiating, and `input` is the input vector fed to the neural network (i.e. the point where we want to differentiate). The jacobian has 3x5 dimensions, that means that to compute the hessian for each `i` I must compute 15 times the same jacobian. That’s not efficient.

Moreover, I’m in fact only interested in what I would call the “second order jacobian” and not the full hessian. That is, the diagonals of the hessian (dy/dx^2). So computing the full hessian is wasteful anyway. In other words, what I would like is to compute the second order derivative of each output of a neural network with respect to each input.

---

_[View the full topic](https://discourse.julialang.org/t/is-there-an-efficient-way-to-compute-the-hessian-of-a-nn/26971)._
