I am using NLopt to minimize (gradient-free) a function f(\theta), where \theta is a vector of parameters. There is a non-trivial combination of these parameters for which f(\theta) is not defined and thus throws an error.
Since this combination of parameters is non-trivial I cannot pass it explicitly as a constraint on the maximization. However, I can test it. In other words, if I pass a particular vector \theta_{1} to the function, I am able to tell whether f(\theta_{1}) is defined or not without evaluating it.
Is there a way to tweak NLOpt so that it ignores these combinations that are problematic.
Are you thinking of a situation where your optimum is close to the region in parameter space where your function is undefined, and this tweak would potentially make the optimizer shy away from it due to the large penalty in the undefined region? Seems theoretically possible, but I don’t know enough about how NLOpt works to guess whether this would be a problem in practice.
It really depends on which optimization algorithm you are using (NLopt supports many) — some of them implicitly assume that your objective function is continuous (or even differentiable etc), and others do not.
Since this combination of parameters is non-trivial I cannot pass it explicitly as a constraint on the maximization.
It is even more difficult than a nonlinear constraint. The constraint is that a matrix generated by the parameters and some data is positive definite. The only solution that comes to mind is that if the matrix is not PD given some guess then the value of the objective function is a very large number.
You could write this as a nonlinear inequality constraint on the smallest eigenvalue, which has the virtue of being continuous and mostly differentiable.
(Have you thought about linearizing your problem into a sequence of SDPs?)
However, note that nonlinear optimization algorithms will still occasionally evaluate points that violate your constraints, so you still want your objective function to give a value at these points, and ideally do so in a way that varies smoothly across the constraint boundary.
In particular, see the PENNON page and references thereof on methods for general nonlinear optimization with semidefinite constraints. They have a free (GPL) matlab implementation which could potentially be ported to Julia, or someone could re-implement one of these algorithms based on the descriptions in the papers.
Alternatively, it might be possible to re-formulate your problem so that your matrices are automatically semidefinite. For example, if you construct a matrix A = B(p)^T B(p) where B(p) is a matrix function of your parameters p, then your matrix A will be automatically semidefinite. Without knowing more about your problem it’s impossible to give you more specific advice.
Most of the practical effort in nonlinear optimization lies in how you formulate the problem rather than in the algorithms. The key is to formulate the problem in a way that allows you to exploit the best available algorithms.
One runs into these kind of problems all the time with complex models in economics — some parameters just don’t have an equilibrium, and it is not practically possible to treat this as a constraint.
In my own packages I am experimenting with allowing objective functions or residuals (for nonlinear solvers) to return nothing in this case — this is a bit more disciplined than 1e20 or NaN. It would be great if there was a standard way of handling this though across various optimization and rootfinding packages.