# NLopt stalls without objective progress?

**URL:** <https://discourse.julialang.org/t/nlopt-stalls-without-objective-progress/54073>\
**Category:** Optimization (Mathematical)\
**Created:** [January 27, 2021, 7:59pm UTC](https://discourse.julialang.org/t/nlopt-stalls-without-objective-progress/54073 "2021-01-27T19:59:02Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![evanfields](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/evanfields/32/1744_2.png) [@evanfields](https://discourse.julialang.org/u/evanfields)\
**Post date:** [January 27, 2021, 7:59pm UTC](https://discourse.julialang.org/t/nlopt-stalls-without-objective-progress/54073/1 "2021-01-27T19:59:02Z")

</div>

I have an optimization problem which I’m solving with NLopt (algorithm `:LD_MMA`). The objective is nonlinear, there are upper/lower bound constraints, and one linear constraint. It seems like no matter how long I let NLopt run (ie setting `maxtime` before optimizing), the entire time budget is used up and the solver reports its stop reason as `:MAXTIME_REACHED` with the same objective returned. Here’s a plot of maxtime vs objective attained (it’s a maximization problem):

```julia
       ┌────────────────────────────────────────┐ 
   320 │⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
       │⢰⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠉⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
       │⢸⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
       │⢸⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
       │⢸⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
       │⡎⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
       │⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
       │⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
       │⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
       │⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
       │⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
       │⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
       │⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
       │⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
   280 │⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
       └────────────────────────────────────────┘ 
       0 20

```

Between 0.5 and 60 seconds of runtime, the objective value found improves by exactly 0. I’m not sure how this can happen, as I have `ftol_abs = .1`; shouldn’t that cause NLopt to stop optimizing when the objective doesn’t increase by this amount?

In contrast, if I run the same problem from a different initial point, a (slightly) better solution is found and NLopt terminates (`:FTOL_REACHED`) in less than one second.

Any hints about what might be causing NLopt to stall at a suboptimal point for initial points, and how to avoid this?

(The problem itself is large and involves some proprietary code, so I’m afraid I can’t post the problem in full. I can answer questions about it though, as needed.)

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [January 27, 2021, 10:08pm UTC](https://discourse.julialang.org/t/nlopt-stalls-without-objective-progress/54073/2 "2021-01-27T22:08:03Z")

</div>

Other people might have a similar experience, but I assume it’s hard to offer advice without the code.

Did you try different algorithms?

What about adding logging to your functions so you can see where it gets stuck? Is it iterating and not terminating? Or is it just stuck on a single iteration for 60 seconds?

---

<div class="post-metadata">

**Author:** ![evanfields](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/evanfields/32/1744_2.png) [@evanfields](https://discourse.julialang.org/u/evanfields)\
**Post date:** [January 30, 2021, 1:13pm UTC](https://discourse.julialang.org/t/nlopt-stalls-without-objective-progress/54073/3 "2021-01-30T13:13:58Z")

</div>

Thank you for the reply. With some logging I learned that my gradient evaluation was hitting some numerical issues and returning all `NaN` after a few iterations. (_Why_ is a question for a separate thread, currently fighting with CUDA…)

What are some algorithms I should try besides `:LD_MMA`? The NLopt documentation says

> Of these algorithms, only MMA and SLSQP support arbitrary nonlinear inequality constraints, and only SLSQP supports nonlinear equality constraints; the rest support bound-constrained or unconstrained problems only. (However, any of them can be applied to nonlinearly constrained problems by combining them with the [augmented Lagrangian method](https://nlopt.readthedocs.io/en/latest/NLopt_Algorithms/#augmented-lagrangian-algorithm) below.)

So even though I have only bound constraints and a few _linear_ constraints, I guess it has to be MMA or SLSQP if I want to use gradient information, which seems likely? Or should I be looking into the Lagrangian method with some other alg?

For context, my problem has about ~1000 decision variables and it takes about ~.2 seconds to evaluate an objective + derivative.