Under exactly what conditions does optimize in Optim.jl return Stopped by an increasing objective: true?

I have a fairly complicated optimisation problem using LBFGS which terminates at some point with this statement. The function provided to the solver computes the discrete gradient of the objective function and I have checked that the gradient is indeed a descent direction. Does the line search algorithm play any role here?

Have you tried constructing an Optim.Options with Optim.Options(allow_f_increases=true)? Also maybe try successive_f_tol=n where n>1? I’m not sure what the latter does though. Worth an experiment maybe?

Does the line search algorithm play any role here?

I’m no expect, but I imagine the line search algorithm plays a role. BackTracking, for example, takes aggressive steps, and then (if the objective increased), it automatically back tracks until it sees a decrease in the objective.

Thanks. This works, although it does not make me feel comfortable. Why is the optimiser thinking my gradient is not a descent direction? Are there tolerances in the line search algorithm that can be tuned to achieve the same effect? At the moment I am using

An off-topic question. I have noticed that when the optimisation results are printed, the norm of the gradient |g(x)| that gets displayed does not match with the norm of the gradient that can be computed afterwards from the optimizer. Any known reasons for this?

No probs. I think the algorithm is getting stuck in a local minimum that does not satisfy your stopping criteria, and setting allow_f_increases=true lets the algorithm dig itself out of a hole.

An off-topic question. I have noticed that when the optimisation results are printed, the norm of the gradient |g(x)| that gets displayed does not match with the norm of the gradient that can be computed afterwards from the optimizer. Any known reasons for this?

Perhaps it displays |g(x)| of the step before the final result? I suppose it wouldn’t bother calculating g if f satisfies a stopping criterion.

I can’t speak fo the linesearch part of all this. I’ve never looked into them.