Optim.jl returns `Stopped by an increasing objective: true`

Specific Domains Optimization (Mathematical)
1

Hi,

Under exactly what conditions does optimize in Optim.jl return Stopped by an increasing objective: true?

I have a fairly complicated optimisation problem using LBFGS which terminates at some point with this statement. The function provided to the solver computes the discrete gradient of the objective function and I have checked that the gradient is indeed a descent direction. Does the line search algorithm play any role here?

2

Have you tried constructing an Optim.Options with Optim.Options(allow_f_increases=true)? Also maybe try successive_f_tol=n where n>1? I’m not sure what the latter does though. Worth an experiment maybe?

3

Does the line search algorithm play any role here?

I’m no expect, but I imagine the line search algorithm plays a role. BackTracking, for example, takes aggressive steps, and then (if the objective increased), it automatically back tracks until it sees a decrease in the objective.

1 Like
4

Thanks. This works, although it does not make me feel comfortable. Why is the optimiser thinking my gradient is not a descent direction? Are there tolerances in the line search algorithm that can be tuned to achieve the same effect? At the moment I am using

res = Optim.optimize(f, g, x0, 
                     LBFGS(m=100, linesearch=LineSearches.BackTracking(), 
                           alphaguess=LineSearches.InitialStatic(scaled=true, alpha=0.1)), 
                     Optim.Options(show_trace=true, allow_f_increases=true, iterations=1000, f_tol=1e-10, x_tol=1e-10))

An off-topic question. I have noticed that when the optimisation results are printed, the norm of the gradient |g(x)| that gets displayed does not match with the norm of the gradient that can be computed afterwards from the optimizer. Any known reasons for this?

5

No probs. I think the algorithm is getting stuck in a local minimum that does not satisfy your stopping criteria, and setting allow_f_increases=true lets the algorithm dig itself out of a hole.

An off-topic question. I have noticed that when the optimisation results are printed, the norm of the gradient |g(x)| that gets displayed does not match with the norm of the gradient that can be computed afterwards from the optimizer. Any known reasons for this?

Perhaps it displays |g(x)| of the step before the final result? I suppose it wouldn’t bother calculating g if f satisfies a stopping criterion.

I can’t speak fo the linesearch part of all this. I’ve never looked into them.

6

Can you post your actual output?