# Simultaneous Jacobian and Hessian computations

**URL:** <https://discourse.julialang.org/t/simultaneous-jacobian-and-hessian-computations/44884>\
**Category:** Numerics\
**Tags:** question, zygote, finitediff\
**Created:** [August 13, 2020, 3:52pm UTC](https://discourse.julialang.org/t/simultaneous-jacobian-and-hessian-computations/44884 "2020-08-13T15:52:16Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![mleprovost](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mleprovost/32/7166_2.png) [@mleprovost](https://discourse.julialang.org/u/mleprovost)\
**Post date:** [August 13, 2020, 3:52pm UTC](https://discourse.julialang.org/t/simultaneous-jacobian-and-hessian-computations/44884/1 "2020-08-13T15:52:16Z")

</div>

Hello,

I would like to compute the Jacobian and the Hessian of a function f: \mathbb{R}^n \xrightarrow{} \mathbb{R}^m with n,m \sim60. This function involves multiple Fourier transforms and nonlinear transformations that make it difficult to derive by hand the Jacobian and Hessian. The Jacobian and Hessian are not sparse =(

I am now using FiniteDiff.jl but getting the Jacobian takes 300ms (Forward differentiation) while a function evaluation takes 2ms. I need to evaluate this Jacobian for 300 input vectors. How can I speed up these computations? I don’t need machine precision results, an error of 10^{-6} is good enough. Can we get at the same time the Jacobian and the Hessian?

I can’t apply Zygote because of the FFT transforms

---

<div class="post-metadata">

**Author:** ![dpsanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dpsanders/32/3573_2.png) [@dpsanders](https://discourse.julialang.org/u/dpsanders)\
**Post date:** [August 13, 2020, 8:34pm UTC](https://discourse.julialang.org/t/simultaneous-jacobian-and-hessian-computations/44884/2 "2020-08-13T20:34:33Z")

</div>

Calculating the jacobian and hessian are expensive operations. Why so you expect to be able to speed them up?

---

<div class="post-metadata">

**Author:** ![dpsanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dpsanders/32/3573_2.png) [@dpsanders](https://discourse.julialang.org/u/dpsanders)\
**Post date:** [August 13, 2020, 8:36pm UTC](https://discourse.julialang.org/t/simultaneous-jacobian-and-hessian-computations/44884/3 "2020-08-13T20:36:05Z")

</div>

What do you need to do with them? Eg if you immediately multiply the jacobian by a vector it should be possible to do that more efficiently.

---

<div class="post-metadata">

**Author:** ![mleprovost](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mleprovost/32/7166_2.png) [@mleprovost](https://discourse.julialang.org/u/mleprovost)\
**Post date:** [August 13, 2020, 9:00pm UTC](https://discourse.julialang.org/t/simultaneous-jacobian-and-hessian-computations/44884/4 "2020-08-13T21:00:14Z")

</div>

Thank you for your answer,

> [@dpsanders](#):
>
> Calculating the jacobian and hessian are expensive operations. Why so you expect to be able to speed them up?

There are sometime tricks that I am not aware of, just asking for ideas =)

With finite differences, I need to evaluate the same function a lot of time, maybe it is worth trying to approximate the function first…

I am looking at a sensitivity analysis problem, and I have to compute terms like h^T h with h = \partial\_{x\_i} \partial\_{x\_j} f(x) and (\partial\_{x\_i} f\_{k}(x))^2, for i, j = 1,\ldots,n and k=1, \ldots, m

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [August 13, 2020, 9:13pm UTC](https://discourse.julialang.org/t/simultaneous-jacobian-and-hessian-computations/44884/5 "2020-08-13T21:13:31Z")

</div>

Why wouldn’t Zygote work with the FFTs? Zygote can differentiate at least over FFTW

---

<div class="post-metadata">

**Author:** ![mleprovost](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mleprovost/32/7166_2.png) [@mleprovost](https://discourse.julialang.org/u/mleprovost)\
**Post date:** [August 13, 2020, 9:34pm UTC](https://discourse.julialang.org/t/simultaneous-jacobian-and-hessian-computations/44884/6 "2020-08-13T21:34:03Z")

</div>

The package that evaluates the function f is PotentialFlow.jl([https://github.com/darwindarak/PotentialFlow.jl](https://github.com/darwindarak/PotentialFlow.jl)) that has been written in-place and Zygote doesn’t support mutating arrays.  
I might need to rewrite part of it to use Zygote.

---

<div class="post-metadata">

**Author:** ![mleprovost](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mleprovost/32/7166_2.png) [@mleprovost](https://discourse.julialang.org/u/mleprovost)\
**Post date:** [August 13, 2020, 9:57pm UTC](https://discourse.julialang.org/t/simultaneous-jacobian-and-hessian-computations/44884/7 "2020-08-13T21:57:45Z")

</div>

Also the cost of getting one component of f is the same as getting all the components, how can I modify FiniteDiff.jl to compute the hessian for all the components of f at the same time?  
.

---

<div class="post-metadata">

**Author:** ![longemen3000](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/longemen3000/32/7298_2.png) [@longemen3000](https://discourse.julialang.org/u/longemen3000)\
**Post date:** [August 13, 2020, 10:15pm UTC](https://discourse.julialang.org/t/simultaneous-jacobian-and-hessian-computations/44884/8 "2020-08-13T22:15:42Z")

</div>

This seems like a use case for ForwardDiff + DiffResults