# How exactly do optimizers reject infinity loss?

**URL:** <https://discourse.julialang.org/t/how-exactly-do-optimizers-reject-infinity-loss/91597>\
**Category:** Optimization (Mathematical)\
**Tags:** diffeq\
**Created:** [December 13, 2022, 11:26am UTC](https://discourse.julialang.org/t/how-exactly-do-optimizers-reject-infinity-loss/91597 "2022-12-13T11:26:08Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![fanny1](https://avatars.discourse-cdn.com/v4/letter/f/a88e4f/32.png) [@fanny1](https://discourse.julialang.org/u/fanny1)\
**Post date:** [December 13, 2022, 11:26am UTC](https://discourse.julialang.org/t/how-exactly-do-optimizers-reject-infinity-loss/91597/1 "2022-12-13T11:26:09Z")

</div>

Hi,

I was reading the documentation and on this page [Frequently Asked Questions](https://docs.sciml.ai/DiffEqDocs/stable/basics/faq/) of DifferentialEquations.jl, in the Stability and Divergence section, it says:

**if you see these instability warnings during a parameter estimation process, this is likely the underlying problem. Simply check `sol.retcode != :Success` and throw an `Inf` cost and most optimizers will reject steps in those parameter regimes!**

I wonder which optimizers will do that? and how exactly do they “reject the steps”? (e.g.: do they simply skip the current step? or do they randomly initialize a new set of parameters?)

Thanks.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [December 13, 2022, 12:17pm UTC](https://discourse.julialang.org/t/how-exactly-do-optimizers-reject-infinity-loss/91597/2 "2022-12-13T12:17:42Z")

</div>

> [@fanny1](#):
>
> I wonder which optimizers will do that? and how exactly do they “reject the steps”? (e.g.: do they simply skip the current step? or do they randomly initialize a new set of parameters?)

They skip the current step and do some line search thing. I know the Optim.jl algorithms support this. I need to dig around to see which others do it well.

---

<div class="post-metadata">

**Author:** ![cvanaret](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cvanaret/32/11594_2.png) [@cvanaret](https://discourse.julialang.org/u/cvanaret)\
**Post date:** [December 13, 2022, 12:43pm UTC](https://discourse.julialang.org/t/how-exactly-do-optimizers-reject-infinity-loss/91597/3 "2022-12-13T12:43:51Z")

</div>

I’m not too familiar with your application, but I can say a few words about optimization methods. Any good optimization solver uses a globalization mechanism (usually line search as Chris mentioned or trust region) to restrict the length of the step when not enough progress is made or when the functions cannot be evaluated at the new point.

For more details, see here: [www.researchgate.net/publication/362254109](http://www.researchgate.net/publication/362254109)

---

<div class="post-metadata">

**Author:** ![fanny1](https://avatars.discourse-cdn.com/v4/letter/f/a88e4f/32.png) [@fanny1](https://discourse.julialang.org/u/fanny1)\
**Post date:** [December 14, 2022, 10:30am UTC](https://discourse.julialang.org/t/how-exactly-do-optimizers-reject-infinity-loss/91597/4 "2022-12-14T10:30:18Z")

</div>

Thank you for your reply!

---

<div class="post-metadata">

**Author:** ![fanny1](https://avatars.discourse-cdn.com/v4/letter/f/a88e4f/32.png) [@fanny1](https://discourse.julialang.org/u/fanny1)\
**Post date:** [December 14, 2022, 10:31am UTC](https://discourse.julialang.org/t/how-exactly-do-optimizers-reject-infinity-loss/91597/5 "2022-12-14T10:31:54Z")

</div>

Thank you for your explanation. I will check this paper out 😃