Hello everybody!
I am looking for an optimization package to solve the following minimization problem:
\min_{\bf x \in \Omega} \frac{1}{2}||{\bf f} ({\bf x})||^2 + \lambda ||{\bf x}||^2,
where \Omega is a bounded domain with both restrictions of type:
{\bf x}_l < {\bf x} < {\bf x}_u\\\,
A_{\mathrm{eq}} {\bf x} = {\bf b}_{\mathrm{eq}}.
As an additional constraint, the Jacobian of my problem is not available and cannot be computed using automatic differentiation.
I am looking for package in Julia similar to MATLAB’s lsqnonlin, which uses Trust Region Reflective (TRR) algorithm, but I have not found any package in Julia that fits completely with all my requirements. Does anybody know if there is one suited for my purpose?
Also, if there is any optimization algorithm better than TRR for my purpose (and is implemented in a Julia package) I would love to hear about it, so I am also open to your suggestion and diving deeper in optimization.
Finally, I would use MATLAB but I already have all the code in Julia, the function evaluation is much faster in Julia and I am planning in uploading the code to HPC cluster where MATLAB is not available.
since you mentioned function evaluation, I wonder what is the function f in the obj.
I’m not aware of a package that can handle the problem as is. I did find Enlsip.jl which may work?
Can you use finite differencing?
Also, as an alternative strategy. You can eliminate the equality constraints by using a null space reformulation. Basically:
- Compute a particular solution x_p = A_{\text{eq}}^{-1} b_{\text{eq}}. Just do
x_p = A_eq \ b_eq
- Find the nullspace of \text{null}(A_{\text{eq}}) = Z. Just do
Z = nullspace(A_eq)
- Now you have x = x_p + Z y, \qquad y \in \mathbb{R}^{n-r}.
- Substitute. You should end up with
\min_{y \in \mathbb{R}^{n-r}} \tfrac{1}{2}\bigl\|f(x_p + Z y)\bigr\|^2 + \lambda\|x_p + Z y\|^2 \quad \text{s.t.}\ \ x_l - x_p \le Z y \le x_u - x_p
Now you do need to be a bit more careful with differentiating. Because now the variable is y, and \frac{\partial x}{\partial y} = Z.
With this form you can use the machinery in NonlinearSolve.jl, because the equality constraint is eliminated (see Nonlinear Problems · NonlinearSolve.jl).