the result of the first one is completely unacceptable
the result of the second one is heavily substandard.
the result of the third one is no-problem, but suggesting that Ipopt may NOT converges to the nearest local optimum from the warm-start point. (Then the importance of the warm-start point is impaired?)
These are all expected behavior. Ipopt is a local optimizer. It does not find global minima, but stationary points. Ipopt will only converge to an optimum if the problem is convex and twice differentiable.
In example one, your starting point with cos(x) is a stationary point.
For the second example, I assume that as a heuristic, Ipopt additionally evaluates the objective at the variable bounds, and it found a better solution so it returned that. Evaluating the variable bounds in the first example did not improve the objective.
For the third example, The starting point is at the edge of the feasible region. Ipopt perturbs the solution into the interior, which lets it escape the local maximum.
I see, but
The cos function is a nice function in that it belongs to C^\infty (having arbitrary orders of continuous derivative). In the first example, Ipopt gives me a local Maximum which cannot survive any slight perturbation. In this case, what is the meaning of specifying Min in JuMP.@objective(NLP, Min, cos(x))?
I donāt think Ipopt can memorize the best solution found so far.
Letās discuss it separately here.
In the third example, my meaning was:
Ipopt returns 3 * pi, which is correct. But clearly pi is the nearest optimality, from the start value 0.0.
One more question: is the usage in my first post correct (as expected)?
Do I have to explicitly pass gradient and Hessian info to Ipopt? or something else?
In the 4th example, the expected local minimum is pi = 3.14, because this is a closer minimum.
In the 5th example, the same applies. But the returned solution only achieves -0.759 (substandard, and it is not reasonably poor).
I have no other comments. This is expected behavior of Ipopt. If you violate the assumption that the problem is convex, there is no guarantee what stationary it will find.
Do I have to explicitly pass gradient and Hessian info to Ipopt? or something else?
Nope. JuMP uses automatic differentiation to compute the various gradients and Hessian oracles.
Talking about ācloser minimumā is not relevant here. IPOPT builds a quadratic local model of the original problem at every iteration, and moving from one iterate to the next depends on the shape of this local model. The local model ressembles the original problem only when weāre close to the current point.
IPOPT is generally quite robust, but probably not as robust as an SQP method. For the first instance, the issue is that IPOPT does not follow feasible directions of negative curvature, but regularizes the problem instead. Therefore, it wonāt notice if you start at a local maximum because the optimality conditions are satisfied.
To answer your questions:
yes, if the problem is unconstrained, IPOPT returns a point that is not worse than the starting point. If the problem is constrained, it depends whether the starting point is feasible and what the numerical tolerance is.
indeed, IPOPT doesnāt guarantee global optimality in the nonconvex case, only local optimality. If you start far from a local solution, progress may be slow until you get to a ābasin of attractionā where the Newton method converges quickly. Most methods are equipped with globalization techniques that ensure that, whatever the initial point, you reach a stationary point.
The interior-point method (IPM) implemented in Uno closely follows the IPOPT algorithm (with fewer bells and whistles), so its performance is in the same ballpark as IPOPT (a tad less robust). On your 3 instances, the Uno IPM produces exactly the same results. Feel free to try it out:
using JuMP, AmplNLWriter, Uno_jll
function Optimizer()
options = String["print_solution=yes", "preset=ipopt"]
return AmplNLWriter.Optimizer(Uno_jll.amplexe, options)
end
model = Model(() -> Optimizer())
@variable(model, -Float64(2 * pi) <= x <= Float64(2 * pi), start = 0.0)
@objective(model, Min, cos(x))
JuMP.optimize!(model)
That said, I have a few ideas on how to improve upon IPOPT. Iām hoping Iāll get to try them out this year
To conclude, I think NLP is a problem-specific problem. Every algorithm has its own field to apply properly.āThe closestā minimum does indeed shouldnāt be expected (e.g. considering minimizing cos(1e6 * pi * x)) . Itās hard for a black-box solver to infer how long of a stepsize is proper.
