I was reading the documentation and on this page Frequently Asked Questions of DifferentialEquations.jl, in the Stability and Divergence section, it says:
if you see these instability warnings during a parameter estimation process, this is likely the underlying problem. Simply check sol.retcode != :Success and throw an Inf cost and most optimizers will reject steps in those parameter regimes!
I wonder which optimizers will do that? and how exactly do they “reject the steps”? (e.g.: do they simply skip the current step? or do they randomly initialize a new set of parameters?)
They skip the current step and do some line search thing. I know the Optim.jl algorithms support this. I need to dig around to see which others do it well.
I’m not too familiar with your application, but I can say a few words about optimization methods. Any good optimization solver uses a globalization mechanism (usually line search as Chris mentioned or trust region) to restrict the length of the step when not enough progress is made or when the functions cannot be evaluated at the new point.
