# Make NLOpt ignore some candidates

**URL:** <https://discourse.julialang.org/t/make-nlopt-ignore-some-candidates/54219>\
**Category:** New to Julia\
**Tags:** optimization\
**Created:** [January 29, 2021, 7:34pm UTC](https://discourse.julialang.org/t/make-nlopt-ignore-some-candidates/54219 "2021-01-29T19:34:59Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![jmcastro2109](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmcastro2109/32/38427_2.png) [@jmcastro2109](https://discourse.julialang.org/u/jmcastro2109)\
**Post date:** [January 29, 2021, 7:34pm UTC](https://discourse.julialang.org/t/make-nlopt-ignore-some-candidates/54219/1 "2021-01-29T19:34:59Z")

</div>

Hey,

I am using NLopt to minimize (gradient-free) a function f(\theta), where \theta is a vector of parameters. There is a non-trivial combination of these parameters for which f(\theta) is not defined and thus throws an error.

Since this combination of parameters is non-trivial I cannot pass it explicitly as a constraint on the maximization. However, I can test it. In other words, if I pass a particular vector \theta\_{1} to the function, I am able to tell whether f(\theta\_{1}) is defined or not without evaluating it.

Is there a way to tweak NLOpt so that it ignores these combinations that are problematic.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [January 29, 2021, 7:38pm UTC](https://discourse.julialang.org/t/make-nlopt-ignore-some-candidates/54219/2 "2021-01-29T19:38:13Z")

</div>

Could you not just add something like

```julia
θ == θ_bad && return 1e6

```

at the top of your function?

---

<div class="post-metadata">

**Author:** ![jmcastro2109](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmcastro2109/32/38427_2.png) [@jmcastro2109](https://discourse.julialang.org/u/jmcastro2109)\
**Post date:** [January 29, 2021, 7:39pm UTC](https://discourse.julialang.org/t/make-nlopt-ignore-some-candidates/54219/3 "2021-01-29T19:39:29Z")

</div>

Yes, that is the solution I am considering. Nonetheless, it confuses me if this can mess up the properties of the optimization algorithms.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [January 29, 2021, 7:42pm UTC](https://discourse.julialang.org/t/make-nlopt-ignore-some-candidates/54219/4 "2021-01-29T19:42:50Z")

</div>

Are you thinking of a situation where your optimum is close to the region in parameter space where your function is undefined, and this tweak would potentially make the optimizer shy away from it due to the large penalty in the undefined region? Seems theoretically possible, but I don’t know enough about how NLOpt works to guess whether this would be a problem in practice.

Maybe just try and see?

---

<div class="post-metadata">

**Author:** ![jmcastro2109](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmcastro2109/32/38427_2.png) [@jmcastro2109](https://discourse.julialang.org/u/jmcastro2109)\
**Post date:** [January 29, 2021, 7:45pm UTC](https://discourse.julialang.org/t/make-nlopt-ignore-some-candidates/54219/5 "2021-01-29T19:45:05Z")

</div>

That’s right. That’s my concern. But I’ll try and see what happens. Thanks!

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 29, 2021, 10:08pm UTC](https://discourse.julialang.org/t/make-nlopt-ignore-some-candidates/54219/6 "2021-01-29T22:08:24Z")

</div>

> [@jmcastro2109](#):
>
> Yes, that is the solution I am considering. Nonetheless, it confuses me if this can mess up the properties of the optimization algorithms.

It really depends on which optimization algorithm you are using (NLopt supports many) — some of them implicitly assume that your objective function is continuous (or even differentiable etc), and others do not.

> Since this combination of parameters is non-trivial I cannot pass it explicitly as a constraint on the maximization.

Sounds like a nonlinear constraint?

---

<div class="post-metadata">

**Author:** ![jmcastro2109](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmcastro2109/32/38427_2.png) [@jmcastro2109](https://discourse.julialang.org/u/jmcastro2109)\
**Post date:** [January 29, 2021, 10:18pm UTC](https://discourse.julialang.org/t/make-nlopt-ignore-some-candidates/54219/7 "2021-01-29T22:18:10Z")

</div>

Thanks for the reply!

I am using Subplex.

It is even more difficult than a nonlinear constraint. The constraint is that a matrix generated by the parameters and some data is positive definite. The only solution that comes to mind is that if the matrix is not PD given some guess then the value of the objective function is a very large number.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 29, 2021, 10:21pm UTC](https://discourse.julialang.org/t/make-nlopt-ignore-some-candidates/54219/8 "2021-01-29T22:21:47Z")

</div>

> [@jmcastro2109](#):
>
> The constraint is that a matrix generated by the parameters and some data is positive definite

You could write this as a nonlinear inequality constraint on the smallest eigenvalue, which has the virtue of being continuous and mostly differentiable.

(Have you thought about linearizing your problem into a sequence of SDPs?)

---

<div class="post-metadata">

**Author:** ![jmcastro2109](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmcastro2109/32/38427_2.png) [@jmcastro2109](https://discourse.julialang.org/u/jmcastro2109)\
**Post date:** [January 29, 2021, 10:24pm UTC](https://discourse.julialang.org/t/make-nlopt-ignore-some-candidates/54219/9 "2021-01-29T22:24:08Z")

</div>

That’s super smart. So I can include the following constraint

```julia
minimum(eigvals(creatingmatrix(theta))>0

```

or something like that?

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 29, 2021, 10:25pm UTC](https://discourse.julialang.org/t/make-nlopt-ignore-some-candidates/54219/10 "2021-01-29T22:25:21Z")

</div>

Yes, but it has to be `>= 0` or some other value.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 29, 2021, 10:28pm UTC](https://discourse.julialang.org/t/make-nlopt-ignore-some-candidates/54219/11 "2021-01-29T22:28:37Z")

</div>

However, note that nonlinear optimization algorithms will still occasionally evaluate points that violate your constraints, so you still want your objective function to give a value at these points, and ideally do so in a way that varies smoothly across the constraint boundary.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 30, 2021, 1:49pm UTC](https://discourse.julialang.org/t/make-nlopt-ignore-some-candidates/54219/12 "2021-01-30T13:49:01Z")

</div>

> [@stevengj](#):
>
> (Have you thought about linearizing your problem into a sequence of SDPs?)

In particular, see the [PENNON page](http://web.mat.bham.ac.uk/kocvara/pennon/) and references thereof on methods for general nonlinear optimization with semidefinite constraints. They have a free (GPL) [matlab implementation](http://web.mat.bham.ac.uk/kocvara/penlab) which could potentially be ported to Julia, or someone could re-implement one of these algorithms based on the descriptions in the papers.

Alternatively, it might be possible to re-formulate your problem so that your matrices are automatically semidefinite. For example, if you construct a matrix A = B(p)^T B(p) where B(p) is a matrix function of your parameters p, then your matrix A will be automatically semidefinite. Without knowing more about your problem it’s impossible to give you more specific advice.

Most of the practical effort in nonlinear optimization lies in **how you formulate the problem** rather than in the algorithms. The key is to formulate the problem in a way that allows you to exploit the best available algorithms.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [January 31, 2021, 12:39pm UTC](https://discourse.julialang.org/t/make-nlopt-ignore-some-candidates/54219/13 "2021-01-31T12:39:41Z")

</div>

One runs into these kind of problems all the time with complex models in economics — some parameters just don’t have an equilibrium, and it is not practically possible to treat this as a constraint.

In my own packages I am experimenting with allowing objective functions or residuals (for nonlinear solvers) to return `nothing` in this case — this is a bit more disciplined than `1e20` or `NaN`. It would be great if there was a standard way of handling this though across various optimization and rootfinding packages.

---

<div class="post-metadata">

**Author:** ![jmcastro2109](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmcastro2109/32/38427_2.png) [@jmcastro2109](https://discourse.julialang.org/u/jmcastro2109)\
**Post date:** [January 31, 2021, 2:02pm UTC](https://discourse.julialang.org/t/make-nlopt-ignore-some-candidates/54219/14 "2021-01-31T14:02:49Z")

</div>

What happens if the function is defined to return nothing in NLOpt. Does it crash?

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [February 1, 2021, 4:21pm UTC](https://discourse.julialang.org/t/make-nlopt-ignore-some-candidates/54219/15 "2021-02-01T16:21:38Z")

</div>

It will probably throw an exception?

---

<div class="post-metadata">

**Author:** ![jmcastro2109](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmcastro2109/32/38427_2.png) [@jmcastro2109](https://discourse.julialang.org/u/jmcastro2109)\
**Post date:** [February 8, 2021, 8:28pm UTC](https://discourse.julialang.org/t/make-nlopt-ignore-some-candidates/54219/16 "2021-02-08T20:28:45Z")

</div>

Currently NLOpt returns a 0 I believe.