# Nonlinear optimization help. Optim, JuMP, something else?
**URL:** https://discourse.julialang.org/t/nonlinear-optimization-help-optim-jump-something-else/47163
**Category:** Optimization (Mathematical)
**Tags:** question, optimization
**Created:** [September 23, 2020, 11:23pm UTC](https://discourse.julialang.org/t/nonlinear-optimization-help-optim-jump-something-else/47163 "2020-09-23T23:23:29Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@dweb](https://discourse.julialang.org/u/dweb)
#### Post date: [September 23, 2020, 11:23pm UTC](https://discourse.julialang.org/t/nonlinear-optimization-help-optim-jump-something-else/47163/1 "2020-09-23T23:23:29Z")
I have a nonlinear optimization problem that I’ve tried solving with scipy. For small scale it (the ‘TNC’ option in scipy minimize, specifically) works well, but it’s too slow when I get to thousands of variables. I’m learning Julia to see if I can find something better here.
Here is an example of a small scale version of the problem:
(Using `ComponentArrays` for convenience)
```julia
k_unr_out = [ 0., 3., 1., 5., 2., 4., 5., 0., 0., 2., 0., 4., 5.,
6., 1., 22., 0., 1., 19., 0., 0., 7., 0., 4., 0., 6.]
k_unr_in = [ 7., 4., 9., 3., 2., 3., 4., 11., 4., 4., 5., 4., 3.,
3., 3., 0., 2., 6., 0., 3., 0., 1., 8., 2., 3., 3.]
k_recip = [ 1., 8., 2., 8., 3., 7., 6., 1., 1., 7., 4., 5., 4.,
2., 9., 3., 2., 6., 6., 2., 25., 6., 4., 3., 3., 6.]
N = length(k_unr_out)
v0 = ComponentArray(x=repeat([0.1], N), y=repeat([0.1], N), z=repeat([0.1], N))
function neg_llhood(v, k_unr_out, k_unr_in, k_recip)
llhood = sum(k_unr_out .* log.(v.x))
llhood += sum(k_unr_in .* log.(v.y))
llhood += sum(k_recip .* log.(v.z))
xy = v.x .* v.y'
zz = v.z .* v.z'
Q = log.(1 .+ xy + xy' + zz)
llhood -= sum(tril(Q, -1))
-llhood
end
f = v -> neg_llhood(v, k_unr_out, k_unr_in, k_recip)
```
Since logarithms are taken for everything, my only constraints are `x_i, y_i, z_i > 0 for all i`. I’d like to use Optim to solve the above but I’m not sure exactly how to do that.
In scipy I was using JAX to do auto differentiation so I’m trying to do that here as well (I could compute the gradient analytically if I need to tho). For box constraints the docs for Optim give this example,
```julia
lower = [1.25, -2.1]
upper = [Inf, Inf]
initial_x = [2.0, 2.0]
inner_optimizer = GradientDescent()
results = optimize(f, g!, lower, upper, initial_x, Fminbox(inner_optimizer))
```
I’m confused by the docs. What is `Fminbox` doing? What is an inner optimizer? How do I specify that I want to use AD?
I tried
```julia
lower = repeat([floatmin()], 3*N)
upper = repeat([Inf], 3*N)
inner_optimizer = GradientDescent()
results = optimize(f, lower, upper, v0, Fminbox(inner_optimizer); autodiff=:forward)
```
And I get
```julia
MethodError: no method matching ldiv!(::ComponentVector{Float64}, ::Optim.InverseDiagonal, ::ComponentVector{Float64})
Closest candidates are:
...
```
I’m not sure if JuMP + a solver is preferred for my given problem. I gave it a try
```julia
model = Model(Ipopt.Optimizer)
@variable(model, x[1:N] >= floatmin(), start=0.1)
@variable(model, y[1:N] >= floatmin(), start=0.1)
@variable(model, z[1:N] >= floatmin(), start=0.1)
@NLobjective(model, Max,
sum(k_unr_out[i] * log(x[i]) + k_unr_in[i] * log(y[i]) + k_recip[i] * log(z[i]) for i in 1:N)
- sum(log(1 + x[i] * y[j] + x[j] * y[i] + z[i] * z[j]) for i in 1:N for j in i+1:N))
optimize!(model)
```
This causes my Jupyter kernel to crash.
I’m a bit lost in how to solve this problem most efficiently. Any help appreciated.
---
### Author:  [@odow](https://discourse.julialang.org/u/odow)
#### Post date: [September 24, 2020, 12:22am UTC](https://discourse.julialang.org/t/nonlinear-optimization-help-optim-jump-something-else/47163/2 "2020-09-24T00:22:14Z")
JuMP + Ipopt works well for me. Note that you need small positive lower bounds on the variables to avoid `log(0)`.
```julia
using JuMP
using Ipopt
k_unr_out = [ 0., 3., 1., 5., 2., 4., 5., 0., 0., 2., 0., 4., 5.,
6., 1., 22., 0., 1., 19., 0., 0., 7., 0., 4., 0., 6.]
k_unr_in = [ 7., 4., 9., 3., 2., 3., 4., 11., 4., 4., 5., 4., 3.,
3., 3., 0., 2., 6., 0., 3., 0., 1., 8., 2., 3., 3.]
k_recip = [ 1., 8., 2., 8., 3., 7., 6., 1., 1., 7., 4., 5., 4.,
2., 9., 3., 2., 6., 6., 2., 25., 6., 4., 3., 3., 6.]
N = length(k_unr_out)
model = Model(Ipopt.Optimizer)
@variable(model, x[1:N] >= 0.001, start = 0.1)
@variable(model, y[1:N] >= 0.001, start = 0.1)
@variable(model, z[1:N] >= 0.001, start = 0.1)
@NLobjective(
model,
Max,
sum(
k_unr_out[i] * log(x[i]) +
k_unr_in[i] * log(y[i]) +
k_recip[i] * log(z[i])
for i in 1:N
)
- sum(
log(1 + x[i] * y[j] + x[j] * y[i] + z[i] * z[j])
for i = 1:N, j = (i + 1):N
)
)
julia> optimize!(model)
This is Ipopt version 3.13.2, running with linear solver mumps.
NOTE: Other linear solvers might be more efficient (see Ipopt documentation).
Number of nonzeros in equality constraint Jacobian...: 0
Number of nonzeros in inequality constraint Jacobian.: 0
Number of nonzeros in Lagrangian Hessian.............: 3081
Total number of variables............................: 78
variables with only lower bounds: 78
variables with lower and upper bounds: 0
variables with only upper bounds: 0
Total number of equality constraints.................: 0
Total number of inequality constraints...............: 0
inequality constraints with only lower bounds: 0
inequality constraints with lower and upper bounds: 0
inequality constraints with only upper bounds: 0
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
0 7.6485452e+02 0.00e+00 1.01e+02 -1.0 0.00e+00 - 0.00e+00 0.00e+00 0
1 5.6576765e+02 0.00e+00 4.92e+01 -1.0 9.82e-02 - 1.00e+00 1.00e+00f 1
2 4.1243350e+02 0.00e+00 2.38e+01 -1.0 1.89e-01 - 1.00e+00 1.00e+00f 1
3 3.2077731e+02 0.00e+00 1.08e+01 -1.0 3.35e-01 - 7.39e-01 1.00e+00f 1
4 2.7086780e+02 0.00e+00 5.04e+00 -1.0 5.72e-01 - 1.00e+00 1.00e+00f 1
5 2.4293161e+02 0.00e+00 2.26e+00 -1.0 9.31e-01 - 8.64e-01 1.00e+00f 1
6 2.2531364e+02 0.00e+00 1.08e+00 -1.0 1.57e+00 - 1.00e+00 1.00e+00f 1
7 2.1391224e+02 0.00e+00 4.79e-01 -1.0 2.56e+00 - 9.21e-01 1.00e+00f 1
8 2.0369603e+02 0.00e+00 1.18e+00 -1.7 4.23e+00 - 7.79e-01 8.83e-01f 1
9 1.9855275e+02 0.00e+00 4.09e-01 -1.7 6.47e+00 - 1.00e+00 1.00e+00f 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
10 1.9484320e+02 0.00e+00 8.00e-01 -1.7 1.05e+01 - 1.00e+00 1.00e+00f 1
11 1.9256323e+02 0.00e+00 4.34e-01 -1.7 1.67e+01 - 1.00e+00 1.00e+00f 1
12 1.9100012e+02 0.00e+00 4.82e-01 -1.7 2.70e+01 - 1.00e+00 1.00e+00f 1
13 1.8989596e+02 0.00e+00 5.62e-01 -1.7 4.30e+01 - 1.00e+00 1.00e+00f 1
14 1.8905100e+02 0.00e+00 5.65e-01 -1.7 6.99e+01 - 1.00e+00 1.00e+00f 1
15 1.8847647e+02 0.00e+00 6.14e-01 -1.7 1.12e+02 - 1.00e+00 1.00e+00f 1
16 1.8812660e+02 0.00e+00 8.63e-01 -1.7 1.90e+02 - 7.43e-01 1.00e+00f 1
17 1.8811339e+02 0.00e+00 6.44e-02 -1.7 4.18e-01 -2.0 1.00e+00 1.00e+00f 1
18 1.8777458e+02 0.00e+00 1.63e+01 -2.5 4.59e+02 - 3.37e-01 1.02e-01f 1
19 1.8726075e+02 0.00e+00 2.49e+00 -2.5 2.80e+02 - 7.58e-01 3.61e-01f 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
20 1.8694821e+02 0.00e+00 1.47e+00 -2.5 3.88e+02 - 1.00e+00 1.00e+00f 1
21 1.8671781e+02 0.00e+00 8.31e-01 -2.5 5.85e+02 - 1.00e+00 1.00e+00f 1
22 1.8661940e+02 0.00e+00 8.84e-01 -2.5 9.77e+02 - 1.00e+00 1.00e+00f 1
23 1.8639436e+02 0.00e+00 9.21e-01 -2.5 1.56e+03 - 1.00e+00 1.00e+00f 1
24 1.8639424e+02 0.00e+00 9.64e-02 -2.5 1.24e-01 -2.5 1.00e+00 1.00e+00f 1
25 1.8639322e+02 0.00e+00 1.30e-03 -2.5 1.70e-01 -3.0 1.00e+00 1.00e+00f 1
26 1.8631537e+02 0.00e+00 2.99e-02 -3.8 9.35e-01 -3.4 1.00e+00 9.26e-01f 1
27 1.8630837e+02 0.00e+00 4.02e-04 -3.8 1.85e+00 -3.9 1.00e+00 1.00e+00f 1
28 1.8630134e+02 0.00e+00 4.08e-03 -5.7 4.22e+00 -4.4 1.00e+00 9.75e-01f 1
29 1.8629600e+02 0.00e+00 4.84e-03 -5.7 9.08e+00 -4.9 1.00e+00 1.00e+00f 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
30 1.8628450e+02 0.00e+00 2.19e-02 -5.7 2.84e+01 -5.3 1.00e+00 1.00e+00f 1
31 1.8625887e+02 0.00e+00 1.18e-01 -5.7 7.65e+01 -5.8 1.00e+00 1.00e+00f 1
32 1.8623095e+02 0.00e+00 5.00e+00 -5.7 2.68e+03 - 1.00e+00 1.91e-01f 2
33 1.8614046e+02 0.00e+00 1.53e+00 -5.7 2.99e+03 - 7.96e-01 1.00e+00f 1
34 1.8604107e+02 0.00e+00 1.06e+00 -5.7 4.48e+03 - 1.00e+00 1.00e+00f 1
35 1.8602667e+02 0.00e+00 4.98e+00 -5.7 7.32e+03 - 9.05e-01 2.41e-01f 2
36 1.8602667e+02 0.00e+00 8.77e-01 -5.7 6.79e-01 -0.9 1.00e+00 2.57e-05f 2
37 1.8595026e+02 0.00e+00 1.20e+00 -5.7 8.49e+03 - 1.00e+00 1.00e+00f 1
38 1.8591162e+02 0.00e+00 1.13e+00 -5.7 1.35e+04 - 1.00e+00 1.00e+00f 1
39 1.8590020e+02 0.00e+00 9.40e-01 -5.7 2.16e+04 - 1.00e+00 2.60e-01f 2
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
40 1.8585862e+02 0.00e+00 9.98e-01 -5.7 2.54e+04 - 1.00e+00 8.11e-01f 1
41 1.8583513e+02 0.00e+00 2.32e+00 -5.7 3.36e+04 - 1.00e+00 6.81e-01f 1
42 1.8581185e+02 0.00e+00 1.10e+00 -5.7 4.47e+04 - 1.00e+00 1.00e+00f 1
43 1.8580976e+02 0.00e+00 1.42e+00 -5.7 6.95e+04 - 1.00e+00 7.43e-02f 1
44 1.8579368e+02 0.00e+00 5.05e-01 -5.7 6.90e+04 - 1.00e+00 1.00e+00f 1
45 1.8577710e+02 0.00e+00 3.51e-01 -5.7 1.02e+05 - 1.00e+00 9.40e-01f 1
46 1.8577088e+02 0.00e+00 4.80e-01 -5.7 1.50e+05 - 1.00e+00 5.00e-01f 2
47 1.8575853e+02 0.00e+00 1.75e-01 -5.7 1.87e+05 - 1.00e+00 1.00e+00f 1
48 1.8575842e+02 0.00e+00 1.84e+00 -5.7 2.80e+05 - 7.97e-01 1.14e-01f 2
49 1.8575799e+02 0.00e+00 6.08e-02 -5.7 4.10e-02 -1.4 1.00e+00 1.19e-01f 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
50 1.8575700e+02 0.00e+00 6.57e-01 -5.7 2.96e+05 - 5.99e-01 5.00e-01f 2
51 1.8574447e+02 0.00e+00 3.73e-02 -5.7 2.48e-02 -1.8 1.00e+00 1.00e+00f 1
52 1.8574444e+02 0.00e+00 2.48e-04 -5.7 7.59e-03 -2.3 1.00e+00 1.00e+00f 1
53 1.8574444e+02 0.00e+00 6.66e-06 -5.7 4.05e-03 -2.8 1.00e+00 1.00e+00f 1
54 1.8574437e+02 0.00e+00 1.05e-05 -8.6 1.91e-02 -3.3 1.00e+00 1.00e+00f 1
55 1.8574437e+02 0.00e+00 6.65e-06 -8.6 3.64e-02 -3.7 1.00e+00 8.93e-01f 1
56 1.8574437e+02 0.00e+00 6.65e-06 -8.6 1.09e-01 -4.2 1.00e+00 1.00e+00f 1
57 1.8574436e+02 0.00e+00 6.65e-06 -8.6 3.28e-01 -4.7 1.00e+00 1.00e+00f 1
58 1.8574434e+02 0.00e+00 6.64e-06 -8.6 9.82e-01 -5.2 1.00e+00 1.00e+00f 1
59 1.8574429e+02 0.00e+00 9.44e-06 -8.6 2.94e+00 -5.6 1.00e+00 1.00e+00f 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
60 1.8574414e+02 0.00e+00 8.62e-05 -8.6 8.77e+00 -6.1 1.00e+00 1.00e+00f 1
61 1.8574368e+02 0.00e+00 7.90e-04 -8.6 2.61e+01 -6.6 1.00e+00 1.00e+00f 1
62 1.8574237e+02 0.00e+00 7.18e-03 -8.6 7.67e+01 -7.1 1.00e+00 1.00e+00f 1
63 1.8574128e+02 0.00e+00 2.76e+00 -8.6 1.27e+06 - 7.86e-02 4.23e-03f 3
64 1.8574128e+02 0.00e+00 9.99e-01 -8.6 1.18e+00 -4.8 1.00e+00 9.54e-07f 21
65 1.8573645e+02 0.00e+00 2.18e-01 -8.6 3.72e+05 - 1.00e+00 5.00e-01f 2
66 1.8573065e+02 0.00e+00 1.43e-01 -8.6 4.65e+05 - 1.00e+00 1.00e+00f 1
67 1.8573054e+02 0.00e+00 1.80e-03 -8.6 8.09e-01 -5.3 1.00e+00 1.00e+00f 1
68 1.8573050e+02 0.00e+00 6.63e-06 -8.6 2.30e+00 -5.8 1.00e+00 1.00e+00f 1
69 1.8573039e+02 0.00e+00 7.46e-05 -8.6 7.32e+00 -6.3 1.00e+00 1.00e+00f 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
70 1.8573009e+02 0.00e+00 6.04e-04 -8.6 2.34e+01 -6.8 1.00e+00 1.00e+00f 1
71 1.8572923e+02 0.00e+00 5.13e-03 -8.6 7.18e+01 -7.2 1.00e+00 1.00e+00f 1
72 1.8572730e+02 0.00e+00 3.99e-02 -8.6 1.95e+02 -7.7 1.00e+00 1.00e+00f 1
73 1.8572321e+02 0.00e+00 4.60e-01 -8.6 6.98e+05 - 1.00e+00 1.00e+00f 1
74 1.8572026e+02 0.00e+00 3.78e-02 -8.6 2.02e+02 -8.2 1.00e+00 1.00e+00f 1
75 1.8571858e+02 0.00e+00 4.89e-01 -8.6 1.05e+06 - 1.00e+00 5.00e-01f 2
76 1.8571814e+02 0.00e+00 1.77e-01 -8.6 1.94e+06 - 6.10e-02 3.99e-02f 5
77 1.8571814e+02 0.00e+00 1.77e-01 -8.6 3.88e-02 -3.2 1.00e+00 2.67e-06f 2
78 1.8571379e+02 0.00e+00 1.00e-01 -8.6 1.33e+06 - 1.00e+00 1.00e+00f 1
79 1.8571377e+02 0.00e+00 6.51e-04 -8.6 6.78e-03 -3.7 1.00e+00 1.00e+00f 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
80 1.8571377e+02 0.00e+00 1.28e-06 -8.6 2.03e-02 -4.2 1.00e+00 1.00e+00f 1
81 1.8571377e+02 0.00e+00 1.28e-06 -8.6 6.08e-02 -4.7 1.00e+00 1.00e+00f 1
82 1.8571377e+02 0.00e+00 1.28e-06 -8.6 1.82e-01 -5.2 1.00e+00 1.00e+00f 1
83 1.8571377e+02 0.00e+00 1.27e-06 -8.6 5.40e-01 -5.6 1.00e+00 1.00e+00f 1
84 1.8571376e+02 0.00e+00 1.23e-06 -8.6 1.58e+00 -6.1 1.00e+00 1.00e+00f 1
85 1.8571373e+02 0.00e+00 6.69e-06 -8.6 4.40e+00 -6.6 1.00e+00 1.00e+00f 1
86 1.8571366e+02 0.00e+00 5.67e-05 -8.6 1.40e+01 -7.1 1.00e+00 1.00e+00f 1
87 1.8571347e+02 0.00e+00 4.69e-04 -8.6 4.51e+01 -7.5 1.00e+00 1.00e+00f 1
88 1.8571292e+02 0.00e+00 4.03e-03 -8.6 1.41e+02 -8.0 1.00e+00 1.00e+00f 1
89 1.8571152e+02 0.00e+00 3.45e-02 -8.6 4.04e+02 -8.5 1.00e+00 1.00e+00f 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
90 1.8571061e+02 0.00e+00 2.62e-01 -8.6 2.00e+06 - 1.00e+00 2.44e-01f 3
91 1.8570871e+02 0.00e+00 3.01e-01 -8.6 2.24e+06 - 8.43e-01 5.00e-01f 2
92 1.8570690e+02 0.00e+00 1.63e-02 -8.6 3.66e+02 -9.0 1.00e+00 1.00e+00f 1
93 1.8570654e+02 0.00e+00 5.92e-01 -8.6 2.80e+06 - 1.00e+00 5.00e-01f 2
94 1.8570353e+02 0.00e+00 2.87e-02 -8.6 3.18e+02 -9.4 1.00e+00 1.00e+00f 1
95 1.8570255e+02 0.00e+00 4.12e-01 -8.6 3.50e+06 - 1.00e+00 2.50e-01f 3
96 1.8570088e+02 0.00e+00 2.40e-02 -8.6 4.11e+02 -9.9 1.00e+00 1.00e+00f 1
97 1.8570077e+02 0.00e+00 8.22e-01 -8.6 3.94e+06 - 1.00e+00 5.00e-01f 2
98 1.8569813e+02 0.00e+00 4.93e-02 -8.6 5.24e+02 -10.4 1.00e+00 1.00e+00f 1
99 1.8569757e+02 0.00e+00 6.06e-01 -8.6 4.92e+06 - 8.40e-01 2.50e-01f 3
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
100 1.8569598e+02 0.00e+00 3.60e-02 -8.6 5.85e+02 -10.9 1.00e+00 1.00e+00f 1
101 1.8569521e+02 0.00e+00 3.21e-01 -8.6 5.53e+06 - 1.00e+00 2.50e-01f 3
102 1.8569435e+02 0.00e+00 6.44e-02 -8.6 8.65e+02 -11.4 1.00e+00 1.00e+00f 1
103 1.8569397e+02 0.00e+00 1.14e+00 -8.6 6.22e+06 - 1.00e+00 1.00e+00f 1
104 1.8569213e+02 0.00e+00 1.13e-01 -8.6 1.42e+03 -11.8 1.00e+00 1.00e+00f 1
105 1.8569157e+02 0.00e+00 5.27e-01 -8.6 9.28e+06 - 5.92e-01 2.50e-01f 3
106 1.8569073e+02 0.00e+00 3.97e-02 -8.6 1.26e+03 -12.3 1.00e+00 1.00e+00f 1
107 1.8569041e+02 0.00e+00 7.81e-01 -8.6 1.04e+07 - 1.00e+00 5.00e-01f 2
108 1.8568944e+02 0.00e+00 5.65e-02 -8.6 1.87e+03 -12.8 1.00e+00 1.00e+00f 1
109 1.8568905e+02 0.00e+00 3.04e-01 -8.6 1.29e+07 - 8.79e-01 2.50e-01f 3
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
110 1.8568856e+02 0.00e+00 9.86e-02 -8.6 2.35e+03 -13.3 1.00e+00 1.00e+00f 1
111 1.8568823e+02 0.00e+00 1.10e+00 -8.6 1.45e+07 - 1.00e+00 1.00e+00f 1
112 1.8568744e+02 0.00e+00 1.43e-01 -8.6 1.65e+04 -13.7 1.00e+00 1.00e+00f 1
113 1.8568704e+02 0.00e+00 3.77e-01 -8.6 2.11e+07 - 4.74e-01 2.50e-01f 3
114 1.8568650e+02 0.00e+00 2.63e-02 -8.6 2.41e+03 -14.2 1.00e+00 1.00e+00f 1
115 1.8568627e+02 0.00e+00 3.93e-01 -8.6 2.34e+07 - 1.00e+00 2.50e-01f 3
116 1.8568595e+02 0.00e+00 1.46e-01 -8.6 1.88e+04 -14.7 1.00e+00 1.00e+00f 1
117 1.8568567e+02 0.00e+00 1.18e+00 -8.6 2.59e+07 - 1.00e+00 1.00e+00f 1
118 1.8568555e+02 0.00e+00 2.92e-01 -8.6 1.43e+05 -15.2 1.00e+00 1.00e+00f 1
119 1.8568463e+02 0.00e+00 9.65e-03 -8.6 2.06e+04 -15.6 1.00e+00 1.00e+00f 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
120 1.8568443e+02 0.00e+00 2.52e-01 -8.6 3.51e+07 - 1.00e+00 1.25e-01f 4
121 1.8568421e+02 0.00e+00 1.56e-01 -8.6 5.86e+04 -16.1 1.00e+00 1.00e+00f 1
122 1.8568404e+02 0.00e+00 1.24e+00 -8.6 3.64e+07 - 1.00e+00 1.00e+00f 1
123 1.8568377e+02 0.00e+00 3.76e-01 -8.6 2.71e+06 -16.6 1.00e+00 5.00e-01f 2
124 1.8568332e+02 0.00e+00 3.59e-01 -8.6 4.28e+07 - 1.00e+00 1.00e+00f 1
125 1.8568316e+02 0.00e+00 3.17e-01 -8.6 4.66e+06 -17.1 1.00e+00 9.92e-02f 4
126 1.8568296e+02 0.00e+00 1.24e-01 -8.6 1.06e+06 -17.6 1.00e+00 1.00e+00f 1
127 1.8568273e+02 0.00e+00 2.11e-01 -8.6 2.51e+06 -18.0 1.00e+00 1.00e+00f 1
128 1.8568266e+02 0.00e+00 2.35e-01 -8.6 1.10e+07 -18.5 1.00e+00 5.00e-01f 2
129 1.8568259e+02 0.00e+00 6.94e-01 -8.6 4.74e+07 -18.1 6.62e-01 2.28e-01f 3
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
130 1.8568234e+02 0.00e+00 1.64e-01 -8.6 1.38e+07 -18.6 1.00e+00 5.00e-01f 2
131 1.8568225e+02 0.00e+00 7.55e-01 -8.6 3.20e+07 -19.0 1.00e+00 5.00e-01f 2
132 1.8568202e+02 0.00e+00 2.00e-01 -8.6 3.09e+06 -17.7 1.00e+00 1.00e+00f 1
133 1.8568192e+02 0.00e+00 6.66e-01 -8.6 6.77e+06 -18.2 1.00e+00 1.00e+00f 1
134 1.8568180e+02 0.00e+00 2.57e-01 -8.6 4.27e+07 -18.7 1.00e+00 2.26e-01f 1
135 1.8568164e+02 0.00e+00 1.85e-01 -8.6 3.75e+07 - 1.00e+00 1.00e+00f 1
136 1.8568159e+02 0.00e+00 6.35e-02 -8.6 3.20e+07 - 1.00e+00 1.00e+00f 1
137 1.8568156e+02 0.00e+00 1.50e-01 -8.6 3.57e+07 - 1.00e+00 8.68e-01f 1
138 1.8568155e+02 0.00e+00 2.96e-02 -8.6 5.01e+07 - 1.00e+00 1.00e+00f 1
139 1.8568154e+02 0.00e+00 9.85e-02 -8.6 4.35e+07 - 1.00e+00 1.00e+00f 1
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls
140 1.8568153e+02 0.00e+00 8.67e-03 -8.6 3.15e+07 - 1.00e+00 1.00e+00f 1
141 1.8568153e+02 0.00e+00 3.18e-03 -8.6 1.17e+07 - 1.00e+00 1.00e+00f 1
142 1.8568153e+02 0.00e+00 4.62e-05 -8.6 1.80e+06 - 1.00e+00 1.00e+00f 1
143 1.8568153e+02 0.00e+00 2.05e-08 -8.6 3.60e+04 - 1.00e+00 1.00e+00f 1
144 1.8568152e+02 0.00e+00 1.00e-03 -9.0 8.35e+07 - 1.00e+00 1.00e+00f 1
145 1.8568151e+02 0.00e+00 5.92e-05 -9.0 6.38e+07 - 1.00e+00 1.00e+00f 1
146 1.8568151e+02 0.00e+00 9.32e-05 -9.0 2.35e+07 - 1.00e+00 1.00e+00f 1
147 1.8568151e+02 0.00e+00 9.82e-07 -9.0 2.22e+06 - 1.00e+00 1.00e+00f 1
148 1.8568151e+02 0.00e+00 2.38e-10 -9.0 1.72e+04 - 1.00e+00 1.00e+00f 1
Number of Iterations....: 148
(scaled) (unscaled)
Objective...............: 7.5000768067322724e+01 1.8568151317638171e+02
Dual infeasibility......: 2.3818402275727155e-10 5.8967889129227490e-10
Constraint violation....: 0.0000000000000000e+00 0.0000000000000000e+00
Complementarity.........: 9.0909090937668046e-10 2.2506619601073186e-09
Overall NLP error.......: 9.0909090937668046e-10 2.2506619601073186e-09
Number of objective function evaluations = 345
Number of objective gradient evaluations = 149
Number of equality constraint evaluations = 0
Number of inequality constraint evaluations = 0
Number of equality constraint Jacobian evaluations = 0
Number of inequality constraint Jacobian evaluations = 0
Number of Lagrangian Hessian evaluations = 148
Total CPU secs in IPOPT (w/o function evaluations) = 0.158
Total CPU secs in NLP function evaluations = 0.148
EXIT: Optimal Solution Found.
```
---
### Author:  [@longemen3000](https://discourse.julialang.org/u/longemen3000)
#### Post date: [September 24, 2020, 12:32am UTC](https://discourse.julialang.org/t/nonlinear-optimization-help-optim-jump-something-else/47163/3 "2020-09-24T00:32:32Z")
> [@dweb](#):
>
> ```julia
> lower = repeat([floatmin()], 3*N)
> upper = repeat([Inf], 3*N)
>
> inner_optimizer = GradientDescent()
> results = optimize(f, lower, upper, v0, Fminbox(inner_optimizer); autodiff=:forward)
>
> ```
i made it work via:
- changing the upper and lower vectors to `ComponentArrays`
- patching optim via defining a `ldiv!` method for `ComponentArrays`
The working code is:
```julia
lower = ComponentArray(x=repeat([floatmin()], N), y=repeat([floatmin()], N), z=repeat([floatmin()], N))
upper = ComponentArray(x=repeat([Inf], N), y=repeat([Inf], N), z=repeat([Inf], N))
Optim.ldiv!(out::ComponentVector, P::Optim.InverseDiagonal, A::ComponentVector) = copyto!(out, A .* P.diag)
inner_optimizer = GradientDescent()
opts = Optim.Options(outer_iterations = 2)
results = optimize(f, lower, upper, v0, Fminbox(inner_optimizer),opts,autodiff=:forward)
```
it returns:
```julia
* Status: failure (reached maximum number of iterations)
* Candidate solution
Final objective value: 1.923543e+02
* Found with
Algorithm: Fminbox with Gradient Descent
* Convergence measures
|x - x'| = 1.01e+00 ≰ 0.0e+00
|x - x'|/|x'| = 1.91e-02 ≰ 0.0e+00
|f(x) - f(x')| = 0.00e+00 ≤ 0.0e+00
|f(x) - f(x')|/|f(x')| = 0.00e+00 ≤ 0.0e+00
|g(x)| = 8.51e-02 ≰ 1.0e-08
* Work counters
Seconds run: 12 (vs limit Inf)
Iterations: 2
f(x) calls: 5294
∇f(x) calls: 5294
```
---
### Author:  [@dweb](https://discourse.julialang.org/u/dweb)
#### Post date: [September 24, 2020, 1:11am UTC](https://discourse.julialang.org/t/nonlinear-optimization-help-optim-jump-something-else/47163/4 "2020-09-24T01:11:46Z")
@odow
Thanks. In my code above I did have small positive lower bounds: `floatmin()`
Tried out yours and it worked. Seems like anything lower than about `1e-9` is causing a crash. Weird.
When I use `1e-8` it works. Here is the solution for x, for example:
```julia
1.0e-8
0.007290631774874027
0.0015940689082409119
0.016124419812656176
0.002324386840220619
0.009408701041340888
0.014141186416612538
1.0e-8
1.0e-8
0.003622059665799834
1.0e-8
0.008444562981428526
0.010041379467192309
0.011468246493959913
0.00164553780292984
3.675460461861131e6
1.0e-8
0.0016481368874902372
1.5709758508294916e6
1.0e-8
3.5883307937084737e-7
0.0188577025351847
1.0e-8
0.0058906658490472125
1.0e-8
0.017933252589371566
```
I notice some of these are taking the lowest allowed value. In the end I doubt not being able to go lower than this affects the solution, though.
Thanks for the help! Now, I wonder how this will fare with N=5000 for a total of 15k variables…
@longemen3000
I’m a bit confused about why the upper and lower vectors need to be ComponentArrays. And, what exactly is `ldiv` and why do we need it to implement it?
Anyway, it’s working now so I can test out different algorithms and see what scales. Thanks!
---
### Author:  [@odow](https://discourse.julialang.org/u/odow)
#### Post date: [September 24, 2020, 1:48am UTC](https://discourse.julialang.org/t/nonlinear-optimization-help-optim-jump-something-else/47163/5 "2020-09-24T01:48:30Z")
> Seems like anything lower than about `1e-9` is causing a crash. Weird.
Solvers use tolerances for floating point comparisons. From memory, Ipopt’s is `10e-8`. It can’t meaningfully distinguish between a lower bound of `0` and a lower bound of `floatmin()`
---
### Author:  [@jonniedie](https://discourse.julialang.org/u/jonniedie)
#### Post date: [October 31, 2020, 5:19am UTC](https://discourse.julialang.org/t/nonlinear-optimization-help-optim-jump-something-else/47163/6 "2020-10-31T05:19:16Z")
I put in a [PR](https://github.com/JuliaNLSolvers/Optim.jl/pull/878) with Optim about the `ldiv!` thing. It’s because `Optim.InverseDiagonal` only has `ldiv!` defined for plain `Array`s in the other argument slots rather than `AbstractArray`s.
Also, I don’t think the upper and lower bound have to be `ComponentArrays`. I was able to run this example without them being.