Optim.jl --- Do all Methods Allow Box Constraints? Should all Work Without Them?

[iwelch]

• I have been experimenting with Optim.jl. Do all optimizers offer box constraints?

• NOTE: All optimizers I tried can work without box constraints, except the brand new SAMIN.

This example failed to use them:

julia> using Optim

julia> myfun(x)= (x[1]-2)^2 + (x[2]-3)^2 + ((x[1]-2)*(x[2]-3))^2;

julia> optimize(myfun, [ 0.0, 0.0 ], fill(6.0, 2), fill(100.0, 2), Optim.Options(iterations=20000))

[ChrisRackauckas]

See Fminbox.

http://julianlsolvers.github.io/Optim.jl/latest/user/minimization/

[iwelch]

thx, chris. I thought I had followed it, but did not look close enough. I wonder whether this is sort of usage buglet, too. See

julia> optimize(myfun, [ 0.0, 0.0 ], fill(6.0, 2), fill(100.0, 2), Optim.Options(iterations=20000))
Results of Optimization Algorithm
 * Algorithm: Nelder-Mead
 * Starting Point: [100.0,100.0]
 * Minimizer: [1.9999890189817244,3.0000111941805336]
 * Minimum: 2.458924e-10
 * Iterations: 58
 * Convergence: true
   *  √(Σ(yᵢ-ȳ)²)/n < 1.0e-08: true
   * Reached Maximum Number of Iterations: false
 * Objective Calls: 113

above happily seems to work but ignores the box constraints. instead, it should do the same thing as the following.

now, omit the Optim.Options, and I get the correct error message:

julia> optimize(myfun, [ 0.0, 0.0 ], fill(6.0, 2), fill(100.0, 2))
ERROR: Initial position must be inside the box

Moreover, with the Options, it can fail altogether. Are Optim.Options incompatible with Fminbox?

julia> optimize(myfun, [ 50.0, 50.0 ], fill(6.0, 2), fill(100.0, 2), Fminbox{NelderMead}(), Optim.Options(iterations=200000))
ERROR: No default objective type for Optim.Fminbox{Optim.NelderMead}() and (myfun, [50.0, 50.0], [6.0, 6.0]).

[Yifan_Liu]

Optim DomainError

I am fitting U.S. Kerosene daily price percentage change with Heston Nandi GARCH model. I use the Optim.optimize function for my MLE but got a domain error. My code using RData, Optim function LoadData() return load("Sample.RData")["Sample"] end function HNGARCH(param, sample = Sample) lambda = param[1] omega = param[2] beta = param[3] alpha = param[4] gamma = param[5] delta = param[6] y = sample[:log_return] r = sample[:rf_r…

I asked a similar question before.

[anriseth]

Fminbox behaves a bit differently than the unconstrained Optim algorithms. In the future we will hopefully manage to merge these approaches in a more intuitive way. I would recommend that you use it with the first-order optimization algorithms, rather than NelderMead etc.

Please read all the instructions under “Box minimization” in the docs carefully.

Regarding the passing of options. In Fminbox, the options to the outer optimization algorithm (which handles the box constraints) are specified with keyword arguments. See the source code. Options to the inner optimizer, such as GradientDescent, or LBFGS, is passed via the keyword argument optimizer_o.

To use box constraints with LBFGS, you can do the following. Note that this will calculate derivatives using finite differences. It tells Fminbox to run 10 outer iterations, and LBFGS to run 2 iterations for each time it is called.

f(x)= (x[1]-2)^2 + (x[2]-3)^2 + ((x[1]-2)*(x[2]-3))^2
x0 = fill(50.0, 2)
res = optimize(f, x0, fill(6.0,2), fill(100.0,2), Fminbox{LBFGS}(); show_trace=true, optimizer_o = Optim.Options(show_trace=true))

[pkofod]

To clean up this interface a bit, it might make sense to make this a dedicated function fminbox(obj, x, lower, upper, inner_method, inner_options, possibly_fminbox_options) or similar…

[iwelch]

piping in from the end user perspective, I think it could also be

Options.optim( min=…, max=… )

with intelligence coded into optimize to handle this appropriately.

/iaw

[pkofod]

I have a plan, stay tuned

[pkofod]

You can take part in the discussion at https://github.com/JuliaNLSolvers/Optim.jl/pull/584 where I’ve moved some things around. (still wip, but you can hopefully see what I aim for)