Optimization convergence requirements

Specific Domains
1

Hello

I have a function from [0,1] to [0,1], which I can shown to always have a unique maximum. I have been using Optim.jl with the default Brent’s method to compute it.
I am a statistician, not a specialist on numerical algorithms. But I still need to confirm that this algorithm will never fail me. I see a reference to Brent’s book and I guess I will have to read parts of it. However, I would like some pointers to where I can find criteria for garanteed convergence. Further, I can also compute the derivate and can provide it to an algorithm requiring it if need be. I don’t know which algorithm to choose. My function is well behaved, I just need to plug into my article a refrence to Optim.jl, the algorithm I use, and a justification as to why the way I optimise is valid. I don’t care about speed of convergence.
If any interest, it is for a maximum likelihood estimate. I’d rather spend my time on its asymtotic behaviour then on numerical calculation aspects, I’m more familiar with the former than the later.

Thanks for your help.

2

Is the function convex? If not, is there only a single relative maximum within the initial bracketing interval? As I understand it, if either of these is true, then convergence is guaranteed with Brent’s method.

3

Thanks a lot
I’m pretty sure the log of the likelihood is convex. I’ll check. But I already know that there are no local maximum. Second derivative is always negative.
Could you point my to a reference for this? I will need to plug it in my text with a reference so I don’t get any challenge on this.

4

A good reference is the book: Brent, Richard P. Algorithms for minimization without derivatives. Dover, 2002. This is a reissue of the original 1973 Prentiss-Hall book.

Brent discusses the convergence of the method in Section 5.5 of the book. He requires the function to be unimodal to guarantee convergence to the global extremum.

5

P.S. The Kindle edition of the book can be purchased from Amazon for about $8 US.