`Optimization.LBFGS` fails to converge while `Optim.NelderMead()` works

General Usage
, ,
10

thanks for the insight, so if I updated your code to increase iterations to 10^5:

    return Optim.optimize(F, p0, LBFGS(), Optim.Options(;iterations=10^5); autodiff=:forward)

I get

Status: failure (line search failed)

 * Candidate solution
    Final objective value:     9.802206e+01

 * Found with
    Algorithm:     L-BFGS

btw, even with more iterations, the result from NelderMead and LBFGS don’t converge:

function optim_fit2(func::T, xs, ys, σs, p0; algo = LBFGS()) where T
    F = function (p)
        cs = func(xs, p)
        return chi2(ys, cs, σs)
    end

    return Optim.optimize(F, p0, algo, Optim.Options(;iterations=5000); autodiff=:forward)
end

sol_nm = optim_fit2(ATLAS_f_6p, xs, ys, sigmas, p0_6para; algo = NelderMead())
sol_lbfgs = optim_fit2(ATLAS_f_6p, xs, ys, sigmas, p0_6para; algo = LBFGS())


julia> sol_nm.minimum, sol_lbfgs.minimum
(98.51662613370482, 98.0220580738921)

julia> sol_nm.minimizer, sol_lbfgs.minimizer
([0.014343016762772673, 0.36918264249014077, 0.08258611548972156, -0.019076422764126248, -0.31125906069802134, -0.044992442492889535], [11207.052120386572, -913.9928401545511, 165.74801872441768, 91.61206638696663, 17.418590874571905, 1.1299718078901382])

if we use BFGS + stopping criteria from Connection between BFGS and ROOT.Minuit. Stopping criteria - #4 by Yuan-Ru-Lin

which is yet another set of minimizer…

julia> sol_lbfgs.minimizer |> print
[3.724592263478385, 0.7652684260382869, 4.556910181502994, 1.2983875079831064, -0.1450263191617602, -0.037556051055012134]

I think this is consistent with what @ChrisRackauckas said about the line search in Optim is “bad” (not sure what it means exactly).