# Gradient of a Hessian dependent loss function

**URL:** <https://discourse.julialang.org/t/gradient-of-a-hessian-dependent-loss-function/31322>\
**Category:** Machine Learning\
**Tags:** flux\
**Created:** [November 20, 2019, 6:27pm UTC](https://discourse.julialang.org/t/gradient-of-a-hessian-dependent-loss-function/31322 "2019-11-20T18:27:30Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![mtgd](https://avatars.discourse-cdn.com/v4/letter/m/3e96dc/32.png) [@mtgd](https://discourse.julialang.org/u/mtgd)\
**Post date:** [November 20, 2019, 6:27pm UTC](https://discourse.julialang.org/t/gradient-of-a-hessian-dependent-loss-function/31322/1 "2019-11-20T18:27:30Z")

</div>

For my application it would be useful to have a loss function that depends on the Hessian of a model.  
Computing the Hessian with respect to the input variable (using Tracker) works fine, but trying to get the gradient with respect to the parameters doesn’t work if matrix multiplication (such as for Dense layers) is involved.

Consider the following example:

```julia
W = param(rand(1, 2))
fn = z -> sum(W * z)^2
hess = z -> Tracker.hessian(fn, z)
Tracker.gradient(() -> sum(hess([1.0, 2.0])), Flux.Params([W]))

```

This throws the error:  
`MethodError: Cannot `convert` an object of type Base.ReshapedArray{Float64,2,Transpose{Float64,Array{Float64,2}},Tuple{Base.MultiplicativeInverses.SignedMultiplicativeInverse{Int64}}} to an object of type Transpose{Float64,Array{Float64,2}}`

Any thoughts or advice appreciated.

I suspect this might be a bug somewhere, but it’s hard to track down where. It would already help to know which package is responsible, to submit an issue.

---

<div class="post-metadata">

**Author:** ![jkbest2](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jkbest2/32/7350_2.png) [@jkbest2](https://discourse.julialang.org/u/jkbest2)\
**Post date:** [November 21, 2019, 6:35pm UTC](https://discourse.julialang.org/t/gradient-of-a-hessian-dependent-loss-function/31322/2 "2019-11-21T18:35:38Z")

</div>

Unfortunately I don’t have anything to add here except that I’m also interested in being able to get gradients of a Hessian-dependent function. In this case it might be worth filing an issue with [`Tracker`](https://github.com/FluxML/Tracker.jl) or trying [`Zygote`](https://github.com/FluxML/Zygote.jl) which is intended to replace `Tracker`.