# How to set max\_iterations for Optim.jl

**URL:** <https://discourse.julialang.org/t/how-to-set-max-iterations-for-optim-jl/61410>\
**Category:** Optimization (Mathematical)\
**Tags:** optim\
**Created:** [May 19, 2021, 4:00am UTC](https://discourse.julialang.org/t/how-to-set-max-iterations-for-optim-jl/61410 "2021-05-19T04:00:50Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [May 19, 2021, 4:00am UTC](https://discourse.julialang.org/t/how-to-set-max-iterations-for-optim-jl/61410/1 "2021-05-19T04:00:50Z")

</div>

I have the following code:

```julia
    lower = [-20, -20, -20, -20, -20, -20.0, -20, -20, -20, -20, -20, -20]
    upper = [20, 20, 20, 20, 20, 20.0, 20, 20, 20, 20, 20, 20]
    initial_x = [-1.4665866297620287, -3.5561543609716884, -5.328280757163652, -5.825432425624137, -4.06758438870819, 1.3365850018520555, 0.34726455461348643, 0.8708697538110506, 1.2180971705802224, 1.077432049937649, 0.20510584981238655, -1.6322406908860976]
    inner_optimizer = BFGS(linesearch=LineSearches.BackTracking(order=3)) # GradientDescent()
    results = optimize(test_initial_condition, lower, upper, initial_x, Fminbox(inner_optimizer), Optim.Options(iterations=10000))
    params=(Optim.minimizer(results))

```

The output is:

```julia
* Status: failure

 * Candidate solution
    Final objective value: 1.750779e-01

 * Found with
    Algorithm: Fminbox with BFGS

 * Convergence measures
    |x - x'| = 7.29e-07 ≰ 0.0e+00
    |x - x'|/|x'| = 7.23e-08 ≰ 0.0e+00
    |f(x) - f(x')| = 0.00e+00 ≤ 0.0e+00
    |f(x) - f(x')|/|f(x')| = 0.00e+00 ≤ 0.0e+00
    |g(x)| = 2.36e+01 ≰ 1.0e-08

 * Work counters
    Seconds run: 10 (vs limit Inf)
    Iterations: 1000
    f(x) calls: 255906
    ∇f(x) calls: 16789

```

This is not too bad, but I know that the minimum is zero, and the solver stops at 0.175, so the result could be better. The solver stops after 1000 Iterations. Why is it ignoring the option: “Optim.Options(iterations=10000)” ?

The complete example can be found at: [https://github.com/ufechner7/KiteViewer/blob/sim/test/test\_optim.jl](https://github.com/ufechner7/KiteViewer/blob/sim/test/test_optim.jl)

---

<div class="post-metadata">

**Author:** ![tomaklutfu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomaklutfu/32/2411_2.png) [@tomaklutfu](https://discourse.julialang.org/u/tomaklutfu)\
**Post date:** [May 19, 2021, 5:13am UTC](https://discourse.julialang.org/t/how-to-set-max-iterations-for-optim-jl/61410/2 "2021-05-19T05:13:28Z")

</div>

This seems like a problem of local minima. As you can see below the objective function stopped decreasing in search direction so the algorithm stopped.

> [@ufechner7](#):
>
> ```julia
> |f(x) - f(x')| = 0.00e+00 ≤ 0.0e+00
> |f(x) - f(x')|/|f(x')| = 0.00e+00 ≤ 0.0e+00
> 
> ```

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [May 19, 2021, 7:10am UTC](https://discourse.julialang.org/t/how-to-set-max-iterations-for-optim-jl/61410/3 "2021-05-19T07:10:27Z")

</div>

I don’t think that this is the case. If I enter the result as starting condition for a new optimization the result improves. Furthermore it says: ` * Status: failure`

I don’t think it would indicate a failure if it found a local minimum.

---

<div class="post-metadata">

**Author:** ![tomaklutfu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomaklutfu/32/2411_2.png) [@tomaklutfu](https://discourse.julialang.org/u/tomaklutfu)\
**Post date:** [May 19, 2021, 7:31am UTC](https://discourse.julialang.org/t/how-to-set-max-iterations-for-optim-jl/61410/4 "2021-05-19T07:31:58Z")

</div>

BFGS method uses Hessian Matrix approximation if not provided. If you feed the result again, obviously this matrix is reset so it may find a search direction with the new hessian prediction([I believe it starts with identity matrix](https://julianlsolvers.github.io/Optim.jl/stable/#algo/lbfgs/)). I think it is failed because the norm of gradient is not small but in the search direction the algorithm cannot find `x'` that `f(x')` is lower than `f(x)`. There are some references for failures in [here](https://julianlsolvers.github.io/Optim.jl/stable/#user/minimization/#notes-on-convergence-flags-and-checks) about convergence but it is not clear to me.

> [@ufechner7](#):
>
> ` |g(x)| = 2.36e+01 ≰ 1.0e-08`

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [May 20, 2021, 10:29am UTC](https://discourse.julialang.org/t/how-to-set-max-iterations-for-optim-jl/61410/5 "2021-05-20T10:29:54Z")

</div>

I am using the solver :LN\_BOBYQA from NLopt now. With this solver I can achieve a final objective value as low as 0.0001699, then it terminates with ROUNDOFF\_LIMITED. But it is slow, it needs about 10 minutes.

See: [https://github.com/ufechner7/KiteViewer/blob/sim/test/test\_nlopt.jl](https://github.com/ufechner7/KiteViewer/blob/sim/test/test_nlopt.jl)

Its a pity that no solver from Optim.jl is able to achieve this accuracy.

---

<div class="post-metadata">

**Author:** ![Gabriel\_Kreindler](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gabriel_kreindler/32/24961_2.png) [@Gabriel\_Kreindler](https://discourse.julialang.org/u/Gabriel_Kreindler)\
**Post date:** [October 1, 2021, 6:04pm UTC](https://discourse.julialang.org/t/how-to-set-max-iterations-for-optim-jl/61410/6 "2021-10-01T18:04:32Z")

</div>

I think that this can be achieved using the `outer_iterations = ...` together with `iterations=...` option. Here is my example:

```julia
optimize(myfunction, theta_lower, theta_upper, theta_initial,
		Fminbox(),
			Optim.Options(outer_iterations = 1500,
						  iterations=10000,
						  show_trace=true,
						  show_every=50,
						  f_tol=my_f_tol,
						  g_tol=my_g_tol))

```