Hi,
Today my Optim package was updated. However, after this update, the optimization doesn’t work anymore. For the optimization I use the Nelder-Mead algorithm. In order to set some boundaries, I use Fminbox, i.e., the optimization call looks like this:
res = optimize(x → calc_mse( x ), lower, upper, x0, Fminbox(NelderMead()) )
Whereas the code was running fine before the update, now he did not find a minimum, and furthermore two issues occur:
i) the resulting value of objective function is larger than the initial one, and
ii) (at least) one component of the fitting parameter exceeds the boundaries.
An example which reproduces my problem is the following (using Julia 1.5.2):
using Optim
function calc_mse(p)
x = p[1]
y = p[2]
return 1 ./ (abs.(sin.(x).^70 .* cos.(y)).^60 .+.01 )
endoffset = rand(2) / 10
x0 = [pi/2, pi] .+ offset
lower = [x0[1] - 0.1, x0[2] - 0.1]
upper = [x0[1] + 0.1, x0[2] + 0.1]
iterationen = 100
res0 = optimize(x → calc_mse( x ), x0, NelderMead(), Optim.Options(iterations = iterationen, show_trace = true, show_every=1) )
res1 = optimize(x → calc_mse( x ), lower, upper, x0, Fminbox(LBFGS()), Optim.Options(iterations = iterationen, show_trace = true, show_every=1) )
res2 = optimize(x → calc_mse( x ), lower, upper, x0, Fminbox(NelderMead()), Optim.Options(iterations = iterationen, show_trace = true, show_every=1) )
The unconstrained version of the Nelder-Mead algorithm works well. The same is true for the optimization with boundaries using the L-BFGS algorithm. Only the last one, which uses the Nelder-Mead algorithm, does not work.
Is this a bug or does something changed and have to be considered in the new version of Optim for the Nelder-Mead algorithm?