# How to properly specify maximum interations in Optimization?

**URL:** <https://discourse.julialang.org/t/how-to-properly-specify-maximum-interations-in-optimization/109144>\
**Category:** Optimization (Mathematical)\
**Tags:** question\
**Created:** [January 23, 2024, 2:45pm UTC](https://discourse.julialang.org/t/how-to-properly-specify-maximum-interations-in-optimization/109144 "2024-01-23T14:45:20Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![foolcat](https://avatars.discourse-cdn.com/v4/letter/f/a3d4f5/32.png) [@foolcat](https://discourse.julialang.org/u/foolcat)\
**Post date:** [January 23, 2024, 2:45pm UTC](https://discourse.julialang.org/t/how-to-properly-specify-maximum-interations-in-optimization/109144/1 "2024-01-23T14:45:20Z")

</div>

Hello,

I am pretty new to Julia and SciML. I have a question about how to specify the maximum iterations in Optimzation.jl. I read the documentation and tried different combinations of maxiters, iterations, etc. So far it does not work. For example, in the following line of code, I tried to set maxiters=400 and iterrations=500. But the optimizer ran to about 1000 iterations so far. These keyward arguments seem ignored. I must understood the documentation wrong some how.

```julia
res = Optimization.solve(optprob, Optim.BFGS(), maxiters=400;iterations=500)

```

Thanks.

---

<div class="post-metadata">

**Author:** ![mike.ingold](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mike.ingold/32/203749_2.png) [@mike.ingold](https://discourse.julialang.org/u/mike.ingold)\
**Post date:** [January 23, 2024, 3:33pm UTC](https://discourse.julialang.org/t/how-to-properly-specify-maximum-interations-in-optimization/109144/2 "2024-01-23T15:33:32Z")

</div>

The [(L-)BFGS - Optim.jl (julianlsolvers.github.io)](https://julianlsolvers.github.io/Optim.jl/stable/#algo/lbfgs/) solver requires the gradient to be calculated at every step. Calculating the gradient requires an additional evaluation of the function being minimized to inform which direction the next guess should be in. So, if you set a particular number N of iterations of the optimization process and use a gradient-required solver, you should expect that function to be evaluated 2N times.

---

<div class="post-metadata">

**Author:** ![foolcat](https://avatars.discourse-cdn.com/v4/letter/f/a3d4f5/32.png) [@foolcat](https://discourse.julialang.org/u/foolcat)\
**Post date:** [January 23, 2024, 4:22pm UTC](https://discourse.julialang.org/t/how-to-properly-specify-maximum-interations-in-optimization/109144/3 "2024-01-23T16:22:31Z")

</div>

Thanks for you quick reply. I am still confused. For (L-)BFGS, the optimization formula is

x\_{n+1} = x\_{n} - P^{-1} \nabla f(x\_n)

What I really want is to put a upper limit on n. My ODE problem f(x) is very expensive to solve. Sometime I just want to get some “good enough” parameter estimation after say n = 100 iterations. “maxiter” does not seem to work.

---

<div class="post-metadata">

**Author:** ![mike.ingold](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mike.ingold/32/203749_2.png) [@mike.ingold](https://discourse.julialang.org/u/mike.ingold)\
**Post date:** [January 23, 2024, 4:51pm UTC](https://discourse.julialang.org/t/how-to-properly-specify-maximum-interations-in-optimization/109144/4 "2024-01-23T16:51:50Z")

</div>

That formula requires knowledge of \nabla f(x\_n), the gradient of your function at some particular parameter x\_n. So, if the solver is currently situated at some point x\_n in parameter space, it needs to calculate the gradient at its local position in order to determine where x\_{n+1} should be. How does the solver calculate this gradient? Commonly:

- Most solvers allow another gradient function \nabla f to be explicitly provided by the user if you have one that is efficient to compute. Using this, each iteration requires one evaluation of f and one evaluation of the provided \nabla f.
- A finite difference method could be used to estimate \nabla f, but now you need to compute at minimum f(x\_n) and f(x\_n+\Delta x) just to get an estimate of the gradient. Using this, each iteration requires at least 3 evaluations of f.
- The default option: use automatic differentiation methods to calculate the actual \nabla f (to approximately within machine precision) for the cost of only a single additional evaluation of f. Using this, each iteration requires only 2 evaluations of f.

If this second evaluation of f is too costly for you, then you’ll need to either figure out a way to compute the gradient more efficiently, or consider selecting an optimization algorithm that doesn’t require gradient: see [the list of Gradient-Free algorithms on the left-side column here](https://julianlsolvers.github.io/Optim.jl/stable/#algo/nelder_mead/).

---

<div class="post-metadata">

**Author:** ![gabo-di](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gabo-di/32/206427_2.png) [@gabo-di](https://discourse.julialang.org/u/gabo-di)\
**Post date:** [January 23, 2024, 10:58pm UTC](https://discourse.julialang.org/t/how-to-properly-specify-maximum-interations-in-optimization/109144/5 "2024-01-23T22:58:59Z")

</div>

I think mike is correct.  
But to elaborate more the answer. I believe that you are using Optim.jl, so you can see the code here [Optim.jl/src/multivariate/optimize/optimize.jl at master · JuliaNLSolvers/Optim.jl · GitHub](https://github.com/JuliaNLSolvers/Optim.jl/blob/master/src/multivariate/optimize/optimize.jl) around line 52

```julia
    while !converged && !stopped && iteration < options.iterations
        iteration += 1
        ls_success = !update_state!(d, state, method)

```

and the `update_state! ` for the solver that you are using, in this case BFGS [Optim.jl/src/multivariate/solvers/first\_order/bfgs.jl at master · JuliaNLSolvers/Optim.jl · GitHub](https://github.com/JuliaNLSolvers/Optim.jl/blob/master/src/multivariate/solvers/first_order/bfgs.jl) around line 141

```julia
    # Update current position
    state.dx .= state.alpha.*state.s
    state.x .= state.x .+ state.dx

```

which is more or less the equation that you wrote. So this part is controlled by `options.iterations`, since the options `iterations` and `maxiters` can overwrite the same variable, depending on the solver that you use, which you can see on you OptimizationOptimJL installed package, in the function `__map_optimizer_args`. For this particular solver the final value in `options.iterations` is given by `maxiters`.

Now, I think the arguments that you care about are `f_calls_limit, g_calls_limit, h_calls_limit`, that are related to mike’s answer. Which you can see them in action here [Optim.jl/src/multivariate/optimize/optimize.jl at master · JuliaNLSolvers/Optim.jl · GitHub](https://github.com/JuliaNLSolvers/Optim.jl/blob/master/src/multivariate/optimize/optimize.jl) around line 79

```julia
        f_limit_reached = options.f_calls_limit > 0 && f_calls(d) >= options.f_calls_limit ? true : false
        g_limit_reached = options.g_calls_limit > 0 && g_calls(d) >= options.g_calls_limit ? true : false
        h_limit_reached = options.h_calls_limit > 0 && h_calls(d) >= options.h_calls_limit ? true : false

```

So do something like

```julia
res = solve(prob, Optim.BFGS(), maxiters=400; f_calls_limit=2000)

```