# Ipopt's "local" optimum can be very substandard

**URL:** <https://discourse.julialang.org/t/ipopts-local-optimum-can-be-very-substandard/127217>\
**Category:** Optimization (Mathematical)\
**Tags:** question, jump, ipopt\
**Created:** [March 21, 2025, 11:05am UTC](https://discourse.julialang.org/t/ipopts-local-optimum-can-be-very-substandard/127217 "2025-03-21T11:05:28Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![WalterMadelim](https://avatars.discourse-cdn.com/v4/letter/w/3e96dc/32.png) [@WalterMadelim](https://discourse.julialang.org/u/WalterMadelim)\
**Post date:** [March 21, 2025, 11:05am UTC](https://discourse.julialang.org/t/ipopts-local-optimum-can-be-very-substandard/127217/1 "2025-03-21T11:05:28Z")

</div>

Continuing the discussion from [The solution deteriorates if the domain of decisions is improperly set when using Ipopt](https://discourse.julialang.org/t/the-solution-deteriorates-if-the-domain-of-decisions-is-improperly-set-when-using-ipopt/126743):

The title is expounded with 3 examples, where

1. the result of the first one is completely unacceptable
2. the result of the second one is heavily substandard.
3. the result of the third one is no-problem, but suggesting that Ipopt may NOT converges to the _nearest_ local optimum from the _warm-start_ point. (Then the importance of the _warm-start_ point is impaired?)

```julia
import JuMP, Ipopt
# 1️⃣
NLP = JuMP.Model(Ipopt.Optimizer) # continuous nonlinear program
JuMP.@variable(NLP, -Float64(2 * pi) <= x <= Float64(2 * pi), start = 0.0)
JuMP.@objective(NLP, Min, cos(x))
JuMP.optimize!(NLP)
JuMP.termination_status(NLP), JuMP.primal_status(NLP) # (MathOptInterface.LOCALLY_SOLVED, MathOptInterface.FEASIBLE_POINT)
x = JuMP.value(x) # 0.0
cos(x) # 1.0
# 2️⃣
NLP = JuMP.Model(Ipopt.Optimizer)
JuMP.@variable(NLP, -10.0 <= x <= 14.6, start = 0.0)
JuMP.@objective(NLP, Min, cos(x))
JuMP.optimize!(NLP)
JuMP.termination_status(NLP), JuMP.primal_status(NLP) # (MathOptInterface.LOCALLY_SOLVED, MathOptInterface.FEASIBLE_POINT)
x = JuMP.value(x) # 14.600000143196766
cos(x) # -0.44648501954346353
# 3️⃣
NLP = JuMP.Model(Ipopt.Optimizer)
JuMP.@variable(NLP, 0.0 <= x <= 15.0, start = 0.0)
JuMP.@objective(NLP, Min, cos(x))
JuMP.optimize!(NLP)
JuMP.termination_status(NLP), JuMP.primal_status(NLP) # (MathOptInterface.LOCALLY_SOLVED, MathOptInterface.FEASIBLE_POINT)
x = JuMP.value(x) # 9.424777960585782
cos(x) # -1.0

```

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [March 21, 2025, 8:38pm UTC](https://discourse.julialang.org/t/ipopts-local-optimum-can-be-very-substandard/127217/2 "2025-03-21T20:38:25Z")

</div>

These are all expected behavior. Ipopt is a local optimizer. It does not find global minima, but stationary points. Ipopt will only converge to an optimum if the problem is convex and twice differentiable.

In example one, your starting point with `cos(x)` is a stationary point.

For the second example, I assume that as a heuristic, Ipopt additionally evaluates the objective at the variable bounds, and it found a better solution so it returned that. Evaluating the variable bounds in the first example did not improve the objective.

For the third example, The starting point is at the edge of the feasible region. Ipopt perturbs the solution into the interior, which lets it escape the local maximum.

---

<div class="post-metadata">

**Author:** ![WalterMadelim](https://avatars.discourse-cdn.com/v4/letter/w/3e96dc/32.png) [@WalterMadelim](https://discourse.julialang.org/u/WalterMadelim)\
**Post date:** [March 21, 2025, 9:58pm UTC](https://discourse.julialang.org/t/ipopts-local-optimum-can-be-very-substandard/127217/3 "2025-03-21T21:58:03Z")

</div>

I see, but  
The `cos` function is a nice function in that it belongs to C^\infty (having arbitrary orders of continuous derivative). In the first example, Ipopt gives me a local Maximum which cannot survive any slight perturbation. In this case, what is the meaning of specifying `Min` in `JuMP.@objective(NLP, Min, cos(x))`?

> [@odow](#):
>
> and it found a better solution so it returned that

I don’t think Ipopt can memorize the **best solution found so far**.  
Let’s discuss it separately [here](https://discourse.julialang.org/t/ipopt-does-not-memorize-a-best-solution-found-so-far/127244).

In the third example, my meaning was:  
Ipopt returns `3 * pi`, which is correct. But clearly `pi` is the nearest optimality, from the start value `0.0`.

One more question: is the usage in my first post correct (as expected)?  
Do I have to explicitly pass gradient and Hessian info to Ipopt? or something else?

---

<div class="post-metadata">

**Author:** ![WalterMadelim](https://avatars.discourse-cdn.com/v4/letter/w/3e96dc/32.png) [@WalterMadelim](https://discourse.julialang.org/u/WalterMadelim)\
**Post date:** [March 21, 2025, 11:13pm UTC](https://discourse.julialang.org/t/ipopts-local-optimum-can-be-very-substandard/127217/4 "2025-03-21T23:13:11Z")

</div>

Let’s see 2 more examples where the start value is regular.

```julia
import JuMP, Ipopt
# 4️⃣
NLP = JuMP.Model(Ipopt.Optimizer)
JuMP.@variable(NLP, 0.0 <= x <= 15.0, start = 0.15)
JuMP.@objective(NLP, Min, cos(x))
JuMP.optimize!(NLP)
JuMP.termination_status(NLP), JuMP.primal_status(NLP) # (MathOptInterface.LOCALLY_SOLVED, MathOptInterface.FEASIBLE_POINT)
x = JuMP.value(x) # 9.424777960585782
cos(x) # -1.0
# 5️⃣
NLP = JuMP.Model(Ipopt.Optimizer)
JuMP.@variable(NLP, 0.0 <= x <= 15.0, start = 0.18)
JuMP.@objective(NLP, Min, cos(x))
JuMP.optimize!(NLP)
JuMP.termination_status(NLP), JuMP.primal_status(NLP) # (MathOptInterface.LOCALLY_SOLVED, MathOptInterface.FEASIBLE_POINT)
x = JuMP.value(x) # 15.000000146136275
cos(x) # -0.7596880078894556

```

In the 4th example, the expected local minimum is `pi = 3.14`, because this is a closer minimum.  
In the 5th example, the same applies. But the returned solution only achieves `-0.759` (substandard, and it is not reasonably poor).

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [March 21, 2025, 11:14pm UTC](https://discourse.julialang.org/t/ipopts-local-optimum-can-be-very-substandard/127217/5 "2025-03-21T23:14:15Z")

</div>

I have no other comments. This is expected behavior of Ipopt. If you violate the assumption that the problem is convex, there is no guarantee what stationary it will find.

> Do I have to explicitly pass gradient and Hessian info to Ipopt? or something else?

Nope. JuMP uses automatic differentiation to compute the various gradients and Hessian oracles.

---

<div class="post-metadata">

**Author:** ![cvanaret](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cvanaret/32/11594_2.png) [@cvanaret](https://discourse.julialang.org/u/cvanaret)\
**Post date:** [March 21, 2025, 11:21pm UTC](https://discourse.julialang.org/t/ipopts-local-optimum-can-be-very-substandard/127217/6 "2025-03-21T23:21:04Z")

</div>

Talking about “closer minimum” is not relevant here. IPOPT builds a quadratic local model of the original problem at every iteration, and moving from one iterate to the next depends on the shape of this local model. The local model ressembles the original problem only when we’re close to the current point.

---

<div class="post-metadata">

**Author:** ![WalterMadelim](https://avatars.discourse-cdn.com/v4/letter/w/3e96dc/32.png) [@WalterMadelim](https://discourse.julialang.org/u/WalterMadelim)\
**Post date:** [March 22, 2025, 2:29am UTC](https://discourse.julialang.org/t/ipopts-local-optimum-can-be-very-substandard/127217/7 "2025-03-22T02:29:28Z")

</div>

Thanks, I take your point.

So the downside is that the algorithm of Ipopt under the default setting is not versatile.  
But the upside includes:

1. at least it will not generate a solution worse than the start point (?)
2. at least it is fast, since it do not need to guarantee global optimality (?)

I’m not very acquainted with Ipopt. I’ll read its doc then.

**PS** do you think your software (UNO) can do better in these examples?

---

<div class="post-metadata">

**Author:** ![cvanaret](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cvanaret/32/11594_2.png) [@cvanaret](https://discourse.julialang.org/u/cvanaret)\
**Post date:** [March 22, 2025, 9:16pm UTC](https://discourse.julialang.org/t/ipopts-local-optimum-can-be-very-substandard/127217/8 "2025-03-22T21:16:28Z")

</div>

IPOPT is generally quite robust, but probably not as robust as an SQP method. For the first instance, the issue is that IPOPT does not follow feasible directions of negative curvature, but regularizes the problem instead. Therefore, it won’t notice if you start at a local maximum because the optimality conditions are satisfied.

To answer your questions:

1. yes, if the problem is unconstrained, IPOPT returns a point that is not worse than the starting point. If the problem is constrained, it depends whether the starting point is feasible and what the numerical tolerance is.
2. indeed, IPOPT doesn’t guarantee global optimality in the nonconvex case, only local optimality. If you start far from a local solution, progress may be slow until you get to a “basin of attraction” where the Newton method converges quickly. Most methods are equipped with globalization techniques that ensure that, whatever the initial point, you reach a stationary point.

The interior-point method (IPM) implemented in Uno closely follows the IPOPT algorithm (with fewer bells and whistles), so its performance is in the same ballpark as IPOPT (a tad less robust). On your 3 instances, the Uno IPM produces exactly the same results. Feel free to try it out:

```julia
using JuMP, AmplNLWriter, Uno_jll

function Optimizer()
   options = String["print_solution=yes", "preset=ipopt"]
   return AmplNLWriter.Optimizer(Uno_jll.amplexe, options)
end
       
model = Model(() -> Optimizer())
@variable(model, -Float64(2 * pi) <= x <= Float64(2 * pi), start = 0.0)
@objective(model, Min, cos(x))
JuMP.optimize!(model)

```

That said, I have a few ideas on how to improve upon IPOPT. I’m hoping I’ll get to try them out this year 🙂

---

<div class="post-metadata">

**Author:** ![WalterMadelim](https://avatars.discourse-cdn.com/v4/letter/w/3e96dc/32.png) [@WalterMadelim](https://discourse.julialang.org/u/WalterMadelim)\
**Post date:** [March 23, 2025, 12:15am UTC](https://discourse.julialang.org/t/ipopts-local-optimum-can-be-very-substandard/127217/9 "2025-03-23T00:15:15Z")

</div>

> [@cvanaret](#):
>
> if the problem is unconstrained

In this case I may go with `Optim.jl` instead.

To conclude, I think NLP is a problem-specific problem. Every algorithm has its own field to apply properly.“The closest” minimum does indeed shouldn’t be expected (e.g. considering minimizing `cos(1e6 * pi * x)`) . It’s hard for a black-box solver to infer how long of a stepsize is proper.