# Enzyme computing the Hessian(?)

**URL:** <https://discourse.julialang.org/t/enzyme-computing-the-hessian/127197>\
**Category:** Optimization (Mathematical)\
**Tags:** enzyme\
**Created:** [March 21, 2025, 12:06am UTC](https://discourse.julialang.org/t/enzyme-computing-the-hessian/127197 "2025-03-21T00:06:16Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![miguelborrero](https://avatars.discourse-cdn.com/v4/letter/m/eb9ed0/32.png) [@miguelborrero](https://discourse.julialang.org/u/miguelborrero)\
**Post date:** [March 21, 2025, 12:06am UTC](https://discourse.julialang.org/t/enzyme-computing-the-hessian/127197/1 "2025-03-21T00:06:16Z")

</div>

Hi there!

It seems like the (fantastic!) package `Enzyme` supports the computation of the Hessian times a vector but not the Hessian itself. Is this just a matter of time? or is the Hessian too expensive as to implement it?

Thanks!

---

<div class="post-metadata">

**Author:** ![wsmoses](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wsmoses/32/26497_2.png) [@wsmoses](https://discourse.julialang.org/u/wsmoses)\
**Post date:** [March 21, 2025, 12:14am UTC](https://discourse.julialang.org/t/enzyme-computing-the-hessian/127197/2 "2025-03-21T00:14:15Z")

</div>

You can compute a hessian by computing

[  
hessian\_vector\_product([1, 0, …]),  
hessian\_vector\_product([0, 1, …]),  
]

If you think this is useful to add as syntactic sugar for folks, please open a PR/issue!

---

<div class="post-metadata">

**Author:** ![miguelborrero](https://avatars.discourse-cdn.com/v4/letter/m/eb9ed0/32.png) [@miguelborrero](https://discourse.julialang.org/u/miguelborrero)\
**Post date:** [March 21, 2025, 12:31am UTC](https://discourse.julialang.org/t/enzyme-computing-the-hessian/127197/3 "2025-03-21T00:31:29Z")

</div>

True, awesome, thanks a lot!

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [March 21, 2025, 6:49am UTC](https://discourse.julialang.org/t/enzyme-computing-the-hessian/127197/4 "2025-03-21T06:49:44Z")

</div>

Do you have reason to believe your Hessian is sparse?

---

<div class="post-metadata">

**Author:** ![miguelborrero](https://avatars.discourse-cdn.com/v4/letter/m/eb9ed0/32.png) [@miguelborrero](https://discourse.julialang.org/u/miguelborrero)\
**Post date:** [March 21, 2025, 2:49pm UTC](https://discourse.julialang.org/t/enzyme-computing-the-hessian/127197/5 "2025-03-21T14:49:27Z")

</div>

I don’t have a reason to believe this. Implementing the iteration over unit basis suggested by William I indeed get a non-sparse Hessian.

Why are you asking though?

---

<div class="post-metadata">

**Author:** ![wsmoses](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wsmoses/32/26497_2.png) [@wsmoses](https://discourse.julialang.org/u/wsmoses)\
**Post date:** [March 21, 2025, 11:29pm UTC](https://discourse.julialang.org/t/enzyme-computing-the-hessian/127197/6 "2025-03-21T23:29:13Z")

</div>

Putting an `@compile` from Reactant.jl on the outside of the Enzyme hessian code can potentially result in a speedup for sparse code (doing propagation of one hot vectors). It also could automatically reduce allocations and add GPU/TPU support. But also if you have a problem where those aren’t the case, it may not help.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [March 22, 2025, 7:26am UTC](https://discourse.julialang.org/t/enzyme-computing-the-hessian/127197/7 "2025-03-22T07:26:07Z")

</div>

> [@miguelborrero](#):
>
> Why are you asking though?

Because in sparse settings there are faster, non-trivial ways to group the columns of the Hessian. But in dense settings, what Billy suggested is essentially optimal, up to batching (propagating several one-hot vectors at once with `BatchDuplicated`).