# A hacky guide to using automatic differentiation in nested optimization problems

**URL:** <https://discourse.julialang.org/t/a-hacky-guide-to-using-automatic-differentiation-in-nested-optimization-problems/39123>\
**Category:** Optimization (Mathematical)\
**Tags:** economics, forwarddiff\
**Created:** [May 8, 2020, 5:29pm UTC](https://discourse.julialang.org/t/a-hacky-guide-to-using-automatic-differentiation-in-nested-optimization-problems/39123 "2020-05-08T17:29:59Z")\
**Posts on this page:** 6\
**Page:** 2

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [October 18, 2020, 8:32pm UTC](https://discourse.julialang.org/t/a-hacky-guide-to-using-automatic-differentiation-in-nested-optimization-problems/39123/21 "2020-10-18T20:32:26Z")

</div>

Not quite yet: getting there. Forward mode works on Optim though.

---

<div class="post-metadata">

**Author:** ![mateuszbaran](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mateuszbaran/32/221842_2.png) [@mateuszbaran](https://discourse.julialang.org/u/mateuszbaran)\
**Post date:** [April 20, 2022, 7:45pm UTC](https://discourse.julialang.org/t/a-hacky-guide-to-using-automatic-differentiation-in-nested-optimization-problems/39123/22 "2022-04-20T19:45:29Z")

</div>

Has there been any progress on this? I’m planning to work on a similar thing soon.

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [April 20, 2022, 10:47pm UTC](https://discourse.julialang.org/t/a-hacky-guide-to-using-automatic-differentiation-in-nested-optimization-problems/39123/23 "2022-04-20T22:47:21Z")

</div>

Check [GitHub - gdalle/ImplicitDifferentiation.jl: Automatic differentiation of implicit functions](https://github.com/gdalle/ImplicitDifferentiation.jl)

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [April 20, 2022, 10:47pm UTC](https://discourse.julialang.org/t/a-hacky-guide-to-using-automatic-differentiation-in-nested-optimization-problems/39123/24 "2022-04-20T22:47:49Z")

</div>

The docs have optimisation examples [Unconstrained optimization · ImplicitDifferentiation.jl](https://gdalle.github.io/ImplicitDifferentiation.jl/dev/examples/1_unconstrained_optimization/).

---

<div class="post-metadata">

**Author:** ![mateuszbaran](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mateuszbaran/32/221842_2.png) [@mateuszbaran](https://discourse.julialang.org/u/mateuszbaran)\
**Post date:** [April 21, 2022, 8:12am UTC](https://discourse.julialang.org/t/a-hacky-guide-to-using-automatic-differentiation-in-nested-optimization-problems/39123/25 "2022-04-21T08:12:02Z")

</div>

Thanks for the link, that looks quite interesting but I’m not sure that’s what I need. When there is no “implicitness” I’d expect there would just be forward and backward rules directly for calls to some form of `optimize`? Is there something like that already done? Or is there a non-obvious problem with that? My optimization problem is, very roughly, \arg \min\_{x}f(x, \arg\min\_{y} g(x, y)).

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/224680_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [February 17, 2023, 3:32pm UTC](https://discourse.julialang.org/t/a-hacky-guide-to-using-automatic-differentiation-in-nested-optimization-problems/39123/26 "2023-02-17T15:32:11Z")

</div>

Sorry for being late to the party, but the idea was that with ImplicitDifferentiation.jl, you can compute gradients of x \longmapsto \arg\min\_y g(x, y), which can then be fed to the outer optimization loop

[Previous page](https://discourse.julialang.org/t/a-hacky-guide-to-using-automatic-differentiation-in-nested-optimization-problems/39123.md?page=1)