# Global Constrained Nonlinear Optimization

**URL:** <https://discourse.julialang.org/t/global-constrained-nonlinear-optimization/111972>\
**Category:** Optimization (Mathematical)\
**Created:** [March 22, 2024, 1:43pm UTC](https://discourse.julialang.org/t/global-constrained-nonlinear-optimization/111972 "2024-03-22T13:43:19Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![zaccazy](https://avatars.discourse-cdn.com/v4/letter/z/5e9695/32.png) [@zaccazy](https://discourse.julialang.org/u/zaccazy)\
**Post date:** [March 22, 2024, 1:43pm UTC](https://discourse.julialang.org/t/global-constrained-nonlinear-optimization/111972/1 "2024-03-22T13:43:19Z")

</div>

I am looking for a global optimizer to solve a nonlinear problem with three variables and two nonlinear inequality constraints. So far, everything I’ve tried delivers local solutions, i.e. solution heavily relies on the initial values. Do you have any recommendations?

---

<div class="post-metadata">

**Author:** ![mtanneau](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mtanneau/32/17787_2.png) [@mtanneau](https://discourse.julialang.org/u/mtanneau)\
**Post date:** [March 22, 2024, 2:19pm UTC](https://discourse.julialang.org/t/global-constrained-nonlinear-optimization/111972/2 "2024-03-22T14:19:46Z")

</div>

Starting with the obvious: this problem is non-convex, right? Otherwise, any local solver will have given you a global optimum already.

The ANSI group at Los Alamos develops and maintains the [Alpine global solver](https://github.com/lanl-ansi/Alpine.jl), which can solve nonconvex problems to global optimality.  
You can access it from [JuMP](https://jump.dev/JuMP.jl/stable/).

Given that your problem has relatively small dimension, interval arithmetic-based methods such as [IntervalOptimisation](https://github.com/JuliaIntervals/IntervalOptimisation.jl) might give you good results as well.

Finally, if your objective and constraints are quadratic functions, Gurobi supports solving those problems to global optimality. Note that any polynomial function can be reformulated into (possibly several) quadratic constraints, e.g., y = x^{3} can be written as y = x z, z = x^{2}.

---

<div class="post-metadata">

**Author:** ![ericsfraga](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericsfraga/32/192_2.png) [@ericsfraga](https://discourse.julialang.org/u/ericsfraga)\
**Post date:** [March 22, 2024, 2:42pm UTC](https://discourse.julialang.org/t/global-constrained-nonlinear-optimization/111972/3 "2024-03-22T14:42:59Z")

</div>

Maybe try [Fresa.jl](https://www.ucl.ac.uk/~ucecesf/fresa.html) which is in the general registry? It has been developed for such problems and can work well. Happy to discuss off-line if you wish.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [March 22, 2024, 2:52pm UTC](https://discourse.julialang.org/t/global-constrained-nonlinear-optimization/111972/4 "2024-03-22T14:52:23Z")

</div>

There is also the ISRES algorithm in NLopt.

---

<div class="post-metadata">

**Author:** ![Vaibhavdixit02](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vaibhavdixit02/32/2916_2.png) [@Vaibhavdixit02](https://discourse.julialang.org/u/Vaibhavdixit02)\
**Post date:** [March 23, 2024, 12:53am UTC](https://discourse.julialang.org/t/global-constrained-nonlinear-optimization/111972/5 "2024-03-23T00:53:53Z")

</div>

You could use PRIMA or Evolutionary both of which have derivative-free global optimizers that support constraints, if you want to avoid rewriting your problem you might want use the Optimization.jl wrappers for them

> **[PRIMA.jl · Optimization.jl](https://docs.sciml.ai/Optimization/stable/optimization_packages/prima/)**
>
> Documentation for Optimization.jl.

> **[Evolutionary.jl · Optimization.jl](https://docs.sciml.ai/Optimization/stable/optimization_packages/evolutionary/)**
>
> Documentation for Optimization.jl.

You could also use a lot of optimizers from the MathOptInterface wrapper [MathOptInterface.jl · Optimization.jl](https://docs.sciml.ai/Optimization/stable/optimization_packages/mathoptinterface/) as well - including ISRES from NLopt as Steven mentioned above.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [March 23, 2024, 1:01am UTC](https://discourse.julialang.org/t/global-constrained-nonlinear-optimization/111972/6 "2024-03-23T01:01:26Z")

</div>

> [@Vaibhavdixit02](#):
>
> You could use PRIMA or Evolutionary both of which have derivative-free global optimizers that support constraints,

PRIMA is only local optimization algorithms, I believe. And Evolutionary doesn’t support nonlinear constraints.

---

<div class="post-metadata">

**Author:** ![Vaibhavdixit02](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vaibhavdixit02/32/2916_2.png) [@Vaibhavdixit02](https://discourse.julialang.org/u/Vaibhavdixit02)\
**Post date:** [March 23, 2024, 1:15am UTC](https://discourse.julialang.org/t/global-constrained-nonlinear-optimization/111972/7 "2024-03-23T01:15:37Z")

</div>

Ah, you are right on PRIMA -`coblya` is local. Evolutionary does support non-linear constraints [Constraints · Evolutionary.jl](https://wildart.github.io/Evolutionary.jl/dev/constraints/)

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [March 23, 2024, 12:51pm UTC](https://discourse.julialang.org/t/global-constrained-nonlinear-optimization/111972/8 "2024-03-23T12:51:58Z")

</div>

> [@Vaibhavdixit02](#):
>
> Evolutionary does support non-linear constraints [Constraints · Evolutionary.jl](https://wildart.github.io/Evolutionary.jl/dev/constraints/)

Only approximately — it asks you to express the constraints as [penalties](https://en.wikipedia.org/wiki/Penalty_method), but it looks like it does so in a way that doesn’t guarantee that the constraints are actually satisfied.

If you add a _finite_ penalty for violating a constraint, as `Evolutionary.PenaltyConstraints` and `Evolutionary.WorstFitnessConstraints` do, then the optimization is likely to converge to a solution that trades off a constraint violation for an improvement in the objective. You only converge to a feasible solution in the limit where you ramp up the penalty coefficient to infinity.

To ensure that you actually satisfy the constraint, one option is a [barrier function](https://en.wikipedia.org/wiki/Barrier_function) that diverges as you approach a constraint, but these have the tradeoff that they prevent you from reaching the boundary of the feasible region (i.e. you can’t reach exact equality for an inequality constraint). Another option is an [augmented Lagrangian method](https://en.wikipedia.org/wiki/Augmented_Lagrangian_method), which adds Lagrange multipliers in addition to a penalty, so that you can converge to a feasible solution with a finite penalty.

The nice thing about the augmented Lagrangian method is that it allows you to apply _any_ unconstrained optimization method (global or local) to a nonlinearly constrained problem. (NLopt includes this technique.)

---

<div class="post-metadata">

**Author:** ![Vaibhavdixit02](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vaibhavdixit02/32/2916_2.png) [@Vaibhavdixit02](https://discourse.julialang.org/u/Vaibhavdixit02)\
**Post date:** [March 23, 2024, 2:54pm UTC](https://discourse.julialang.org/t/global-constrained-nonlinear-optimization/111972/9 "2024-03-23T14:54:40Z")

</div>

> The nice thing about the augmented Lagrangian method is that it allows you to apply _any_ unconstrained optimization method (global or local) to a nonlinearly constrained problem. (NLopt includes this technique.)

Yes, I plan to support creating the augmented lagrangian in Optimization.jl natively it should be pretty interesting to be able to use a much bigger set of optimizers with non-linear constraints than now.

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [March 23, 2024, 3:12pm UTC](https://discourse.julialang.org/t/global-constrained-nonlinear-optimization/111972/10 "2024-03-23T15:12:55Z")

</div>

Percival.jl is also a native Julia implementation of the augmented Lagrangian algorithm.

---

<div class="post-metadata">

**Author:** ![mateuszbaran](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mateuszbaran/32/221842_2.png) [@mateuszbaran](https://discourse.julialang.org/u/mateuszbaran)\
**Post date:** [April 6, 2024, 11:02am UTC](https://discourse.julialang.org/t/global-constrained-nonlinear-optimization/111972/11 "2024-04-06T11:02:29Z")

</div>

For reference, Manopt also has ALM: [Augmented Lagrangian Method · Manopt.jl](https://manoptjl.org/stable/solvers/augmented_Lagrangian_method/)

---

<div class="post-metadata">

**Author:** ![pierre-haessig](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pierre-haessig/32/217129_2.png) [@pierre-haessig](https://discourse.julialang.org/u/pierre-haessig)\
**Post date:** [July 11, 2024, 7:41am UTC](https://discourse.julialang.org/t/global-constrained-nonlinear-optimization/111972/12 "2024-07-11T07:41:50Z")

</div>

> [@mtanneau](#):
>
> Given that your problem has relatively small dimension, interval arithmetic-based methods such as [IntervalOptimisation](https://github.com/JuliaIntervals/IntervalOptimisation.jl) might give you good results as well.

Thanks for the reference. It seems they currently only support _unconstrained_ optimization though.