Optim.jl vs scipy.optimize once again

General Usage Performance
38

Well, for the sake of information. I tested the Optim LBFGS in comparison with the SPGBox method in a simpler example, and got the following, to reach the same solution:

lbfgs:  15.924 ms (8020 allocations: 9.00 MiB)
spgbox:  44.655 μs (12 allocations: 37.25 KiB)

julia> function run()

         f(x) = sum(sin(v) for v in x)

         x0 = rand(100)

         lower = zeros(length(x0)) .- 0.5
         upper = ones(length(x0))

         g! = (g,x) -> ForwardDiff.gradient!(g,f,x)
         
         r_optim = optimize(f, g!, lower, upper, x0, Fminbox(LBFGS()))
         println(r_optim)

         r_spgbox = spgbox!(f, g!, x0, lower=lower, upper=upper)
         println(r_spgbox)

         @test r_optim.minimizer ≈ x0

         print("lbfgs:"); @btime optimize($f, $g!, $lower, $upper, x0, Fminbox(LBFGS())) setup=(x0=rand(100)) evals=1
         print("spgbox:"); @btime spgbox!($f, $g!, x0, lower=$lower, upper=$upper) setup=(x0=rand(100)) evals=1

         nothing
       end
run (generic function with 1 method)


julia> run()
 * Status: success

 * Candidate solution
    Final objective value:     -4.794255e+01

 * Found with
    Algorithm:     Fminbox with L-BFGS

 * Convergence measures
    |x - x'|               = 1.71e-06 ≰ 0.0e+00
    |x - x'|/|x'|          = 3.43e-07 ≰ 0.0e+00
    |f(x) - f(x')|         = 0.00e+00 ≤ 0.0e+00
    |f(x) - f(x')|/|f(x')| = 0.00e+00 ≤ 0.0e+00
    |g(x)|                 = 1.71e-10 ≤ 1.0e-08

 * Work counters
    Seconds run:   0  (vs limit Inf)
    Iterations:    3
    f(x) calls:    883
    ∇f(x) calls:   883


 SPGBOX RESULT: 

 Convergence achieved. 

 Final objective function value = -47.942553860420375
 Best solution found = [ -0.5, -0.5, -0.5, ..., -0.5]
 Projected gradient norm = 0.0

 Number of iterations = 2
 Number of function evaluations = 3

lbfgs:  15.924 ms (8020 allocations: 9.00 MiB)
spgbox:  44.655 μs (12 allocations: 37.25 KiB)

ps: SPG is a very simple algorithm, and that implementation is in pure Julia. If anyone wants to contribute to making it really available to others, you are all welcome. That implementation is a simple translation from another Fortran code, probably there are some things that can be written better in a more Julian style. Breaking changes in the API can be done, no package is currently depending on that.

3 Likes