# How to force Flux to use FiniteDiff

**URL:** <https://discourse.julialang.org/t/how-to-force-flux-to-use-finitediff/76324>\
**Category:** Machine Learning\
**Tags:** flux, finitediff\
**Created:** [February 13, 2022, 2:14am UTC](https://discourse.julialang.org/t/how-to-force-flux-to-use-finitediff/76324 "2022-02-13T02:14:14Z")\
**Posts on this page:** 1\
**Showing post:** 16

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [February 16, 2022, 10:40pm UTC](https://discourse.julialang.org/t/how-to-force-flux-to-use-finitediff/76324/16 "2022-02-16T22:40:00Z")

</div>

> [@BMval](#):
>
> The loss() function has a maximization problem inside it. (I use Opitm.jl). The solution is differentiable, so numerical derivative works fine, but it looks like Zygote can’t find analytical derivative (if I understand it correctly).

If you are doing a bilevel optimization (optimizing a function that itself solves an optimization problem), you can declare your own `rrule` (vector–Jacobian product) to tell Zygote how to differentiate it efficiently using the implicit-function theorem. (Basically, you differentiate using the KKT conditions describing your inner optimum.)

In general, AD tools need a bit of “help” whenever the function you are differentiating solves a problem approximately by an iterative method (e.g. Newton iterations for root finding, or iterative optimization algorithms, or adaptive quadrature) — even if AD can analyze the iterations, it will end up wasting a lot of effort trying to exactly differentiate the error in your approximation.

See also [Differentiating optimization problem solutions in Julia](https://discourse.julialang.org/t/differentiating-optimization-problem-solutions-in-julia/75988)

---

_[View the full topic](https://discourse.julialang.org/t/how-to-force-flux-to-use-finitediff/76324)._