Convergence in Optim questions

Specific Domains Optimization (Mathematical)
,
1

I have a few questions regarding convergence in Optim:

  1. When an optimization finishes and prints the convergence report, at the top it says either “success” or “failure”. Does this refer to whether or not the algorithm converged (within the specified time, iteration, and function call limits) ?

My next two questions concern the following example using the Rosenbrock function:

f(x) = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2
once_diff = OnceDifferentiable(f, [1.98,-1.95]; autodiff = :forward);
res = Optim.optimize(once_diff, [1.25, -2.1], [Inf, Inf], [2.0, 2.0], Fminbox(ConjugateGradient()),
                     Optim.Options(x_abstol = 1e-3, x_reltol = 1e-3, f_abstol = 1e-3, f_reltol = 
                                   1e-3, g_tol = 1e-3))

This gives the following convergence report:

 Status: failure

 * Candidate solution
    Final objective value:     9.339344e-01

 * Found with
    Algorithm:     Fminbox with Conjugate Gradient

 * Convergence measures
    |x - x'|               = 4.73e-04 ≤ 1.0e-03
    |x - x'|/|x'|          = 1.09e-04 ≤ 1.0e-03
    |f(x) - f(x')|         = 0.00e+00 ≤ 1.0e-03
    |f(x) - f(x')|/|f(x')| = 0.00e+00 ≤ 1.0e-03
    |g(x)|                 = 5.05e-01 ≰ 1.0e-03

 * Work counters
    Seconds run:   0  (vs limit Inf)
    Iterations:    1000
    f(x) calls:    2033
    ∇f(x) calls:   1021
  1. Why does it not indicate the reason for the failure?

I was looking at the Optim source code and it appears to me, based on the function Base.show(io::IO, r::MultivariateOptimizationResults) whose definition starts on line 223, that it’s supposed to print a “failure string” which indicates the reason for the failure. In the case of the above example, why didn’t it print “failure (reached maximum number of iterations)”?

  1. I then checked the convergence:
println(res.x_converged)
println(res.f_converged)
println(res.g_converged)

and got false for all three. The third false makes sense, but why are res.x_converged and res.f_converged both false? In the convergence report, it clearly shows that all convergence criteria except for |g(x)| are met…I also checked res.x_abschange, res.x_relchange, res.f_abschange, and res.f_relchange, just to be sure, and indeed they are all the same value as shown in the report…So I’m pretty confused by this…

1 Like
2

It may be better to make these comments in an issue so that they are not lost.

2 Likes
3

@Tamas_Papp Thanks, I just opened an issue.

Also, this is the first issue I’ve ever opened and I was wondering if there are some general guidelines for how to write an issue. I couldn’t find any on this site, so I made a post here: Guidelines for writing an issue? - Meta Discussion - JuliaLang

Lastly, my first question isn’t an issue…any clarification on Optim’s definition of"success"/“failure” would be helpful!

4

@Michael_Barmann,

The general rules for an issue are the same as Discourse: explain what you did (with a reproducible example), what happened, and what you expected to happen.

The distinction is that the Github issues are generally left for bug reports or feature requests, rather than “help, how do I do X”.

In this case, it seems like a feature request for improved clarity on the reason for failures.

why didn’t it print “failure (reached maximum number of iterations)”

Your suggestion seems like it would be a big improvement.

4 Likes