Problem in Optim.jl using Fminbox and f_calls_limit

Specific Domains Optimization (Mathematical)
1

I have found a problem in Optim.jl using Fminbox and f_calls_limit, it is not well respected the limit.

For instead, using the example for the documentation:

julia> using Optim
julia> f(x) = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2;
julia> x0 = [0.0, 0.0];
julia> optimize(f, x0, LBFGS())
 * Status: success

 * Candidate solution
    Final objective value:     5.378405e-17

 * Found with
    Algorithm:     L-BFGS

 * Convergence measures
    |x - x'|               = 4.54e-11 ≰ 0.0e+00
    |x - x'|/|x'|          = 4.54e-11 ≰ 0.0e+00
    |f(x) - f(x')|         = 2.85e-19 ≰ 0.0e+00
    |f(x) - f(x')|/|f(x')| = 5.29e-03 ≰ 0.0e+00
    |g(x)|                 = 1.21e-13 ≤ 1.0e-08

 * Work counters
    Seconds run:   0  (vs limit Inf)
    Iterations:    24
    f(x) calls:    67
    ∇f(x) calls:   67

If a want to restrict the f(x) calls I can use f_calls_limit in Options:

julia> optimize(f, x0, LBFGS(), Optim.Options(f_calls_limit=10))
 * Status: failure

 * Candidate solution
    Final objective value:     8.415971e-01

 * Found with
    Algorithm:     L-BFGS

 * Convergence measures
    |x - x'|               = 8.55e-02 ≰ 0.0e+00
    |x - x'|/|x'|          = 1.00e+00 ≰ 0.0e+00
    |f(x) - f(x')|         = 1.58e-01 ≰ 0.0e+00
    |f(x) - f(x')|/|f(x')| = 1.88e-01 ≰ 0.0e+00
    |g(x)|                 = 1.58e+00 ≰ 1.0e-08

 * Work counters
    Seconds run:   0  (vs limit Inf)
    Iterations:    1
    f(x) calls:    10
    ∇f(x) calls:   10

However, if a want to restrict the area this f_calls_limit is less respected:

julia> optimize(f, [-5., -5], [5., 5.], x0, Fminbox(LBFGS()), Optim.Options(f_calls_limit=10))
 * Status: success

 * Candidate solution
    Final objective value:     5.377764e-17

 * Found with
    Algorithm:     Fminbox with L-BFGS

 * Convergence measures
    |x - x'|               = 5.23e-07 ≰ 0.0e+00
    |x - x'|/|x'|          = 3.70e-07 ≰ 0.0e+00
    |f(x) - f(x')|         = 0.00e+00 ≤ 0.0e+00
    |f(x) - f(x')|/|f(x')| = 0.00e+00 ≤ 0.0e+00
    |g(x)|                 = 3.48e-10 ≤ 1.0e-08

 * Work counters
    Seconds run:   0  (vs limit Inf)
    Iterations:    8
    f(x) calls:    81
    ∇f(x) calls:   81

However, if I change f_calls_limit the calls is updated, I am not sure if it depends on seed random, but if I does not change f_calls_limit is maintained fixed.

julia> optimize(f, [-5., -5], [5., 5.], x0, Fminbox(LBFGS()), Optim.Options(f_calls_limit=40))
 * Status: success

 * Candidate solution
    Final objective value:     5.850631e-17

 * Found with
    Algorithm:     Fminbox with L-BFGS

 * Convergence measures
    |x - x'|               = 6.99e-07 ≰ 0.0e+00
    |x - x'|/|x'|          = 4.94e-07 ≰ 0.0e+00
    |f(x) - f(x')|         = 0.00e+00 ≤ 0.0e+00
    |f(x) - f(x')|/|f(x')| = 0.00e+00 ≤ 0.0e+00
    |g(x)|                 = 2.64e-10 ≤ 1.0e-08

 * Work counters
    Seconds run:   0  (vs limit Inf)
    Iterations:    3
    f(x) calls:    57
    ∇f(x) calls:   57

I am not sure if there is a better way, but I cannot find it.

Thanks in advance.

2

With bounds on the parameters you have an outer loop as well with the solver’s iterations Configurable Options · Optim lists some options you can use adjust the two levels of kwargs for adjusting the behaviour

1 Like
3

Thanks a lot. You have reason, using outer_iterations to 1, it is working:

julia> optimize(f, [-5., -5], [5., 5.], x0, Fminbox(LBFGS()), Optim.Options(outer_iterations=1, f_calls_limit=10))

 * Status: failure

 * Candidate solution
    Final objective value:     3.036937e-01

 * Found with
    Algorithm:     Fminbox with L-BFGS

 * Convergence measures
    |x - x'|               = 4.96e-01 ≰ 0.0e+00
    |x - x'|/|x'|          = 1.00e+00 ≰ 0.0e+00
    |f(x) - f(x')|         = 0.00e+00 ≤ 0.0e+00
    |f(x) - f(x')|/|f(x')| = 0.00e+00 ≤ 0.0e+00
    |g(x)|                 = 1.62e+00 ≰ 1.0e-08

 * Work counters
    Seconds run:   0  (vs limit Inf)
    Iterations:    1
    f(x) calls:    11
    ∇f(x) calls:   11