# Second-order autodiff: which combinations should work?

**URL:** <https://discourse.julialang.org/t/second-order-autodiff-which-combinations-should-work/114892>\
**Category:** Optimization (Mathematical)\
**Tags:** zygote, forwarddiff, autodiff, enzyme, diffractor\
**Created:** [May 29, 2024, 6:19am UTC](https://discourse.julialang.org/t/second-order-autodiff-which-combinations-should-work/114892 "2024-05-29T06:19:53Z")\
**Posts on this page:** 1\
**Showing post:** 10

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [May 29, 2024, 2:05pm UTC](https://discourse.julialang.org/t/second-order-autodiff-which-combinations-should-work/114892/10 "2024-05-29T14:05:53Z")

</div>

If you want the gradient of a scalar-valued function that depends on the gradient of another scalar-valued function, you can use forward-over-reverse combining ForwardDiff with e.g. Zygote or Enzyme or ReverseDiff. See:

> [@Nested AD with Lux etc](https://discourse.julialang.org/t/nested-ad-with-lux-etc/113573/27):
>
> For future reference, here is a manual implementation of forward-over-reverse calculation with parameter gradients (mixed second derivatives). In particular, if you have a scalar-valued h(x,p) = g(\nabla\_x f) for some scalar-valued f(x,p), then one can similarly derive: \left. \nabla\_p h \right|\_{x,p} = \left. \frac{\partial}{\partial\alpha} \left. \nabla\_p f \right|\_{x + \alpha \left. \nabla g \right|\_{z},p} \right|\_{\alpha = 0} \, , where z = \left. \nabla\_x f \right|\_{x,p}. Here is an e…

(You can also use this approach for general Hessians, but it was less obvious to me that it is efficient for scalar-valued functions.)

---

_[View the full topic](https://discourse.julialang.org/t/second-order-autodiff-which-combinations-should-work/114892)._