I have a few questions regarding convergence in Optim:
- When an optimization finishes and prints the convergence report, at the top it says either “success” or “failure”. Does this refer to whether or not the algorithm converged (within the specified time, iteration, and function call limits) ?
My next two questions concern the following example using the Rosenbrock function:
f(x) = (1.0 - x)^2 + 100.0 * (x - x^2)^2 once_diff = OnceDifferentiable(f, [1.98,-1.95]; autodiff = :forward); res = Optim.optimize(once_diff, [1.25, -2.1], [Inf, Inf], [2.0, 2.0], Fminbox(ConjugateGradient()), Optim.Options(x_abstol = 1e-3, x_reltol = 1e-3, f_abstol = 1e-3, f_reltol = 1e-3, g_tol = 1e-3))
This gives the following convergence report:
Status: failure * Candidate solution Final objective value: 9.339344e-01 * Found with Algorithm: Fminbox with Conjugate Gradient * Convergence measures |x - x'| = 4.73e-04 ≤ 1.0e-03 |x - x'|/|x'| = 1.09e-04 ≤ 1.0e-03 |f(x) - f(x')| = 0.00e+00 ≤ 1.0e-03 |f(x) - f(x')|/|f(x')| = 0.00e+00 ≤ 1.0e-03 |g(x)| = 5.05e-01 ≰ 1.0e-03 * Work counters Seconds run: 0 (vs limit Inf) Iterations: 1000 f(x) calls: 2033 ∇f(x) calls: 1021
- Why does it not indicate the reason for the failure?
I was looking at the Optim source code and it appears to me, based on the function
Base.show(io::IO, r::MultivariateOptimizationResults) whose definition starts on line 223, that it’s supposed to print a “failure string” which indicates the reason for the failure. In the case of the above example, why didn’t it print “failure (reached maximum number of iterations)”?
- I then checked the convergence:
println(res.x_converged) println(res.f_converged) println(res.g_converged)
false for all three. The third false makes sense, but why are
res.f_converged both false? In the convergence report, it clearly shows that all convergence criteria except for |g(x)| are met…I also checked
res.f_relchange, just to be sure, and indeed they are all the same value as shown in the report…So I’m pretty confused by this…