In a model for biology I used a code written in f90 that worked fine, but for greater readability by the future readers of the corresponding paper, I translated it into Julia. In my f90 code, I used minimization routines by Nelder-Mead and Powell (Powell was actually better) of the 1960-70 to deter…

[image] stevengj: The reason why you might want such a check in Nelder–Mead is that the Nelder–Mead algorithm is known to be flawed : it might not converge to a local minimum. This is more annedoctal than anything, but BFGS methods can also fail to converge (and are pretty much the standard m…

[image] lmiq: BFGS methods can also fail to converge (and are pretty much the standard methods for large problems because are good enough in practice): The BFGS method with exact line searches fails for non-convex objective functions | SpringerLink . It’s been known for a couple of decades how…

Yes sure, but the Nelder Mead result is also restricted to a standard implementation, as you mentioned above (I don’t know what exactly is implemented in Optim.jl). I was just pointing that such theoretical results may not be fundamental in practice (for their exceptionality or for the actual implem…

[image] lmiq: such theoretical results may not be fundamental in practice I disagree. Sometimes one needs a certain guarantee that a local minimum is found (or a saddle, or whatever you’re looking for).

Yes, of course. But Nelder Mead variants with convergence guarantees also exist. (The Nelder Mead implemented in Optim.jl is a more modern improvement, indeed. It doesn’t seem to have convergence results, though, but behaves better than the classical method)

I agree. For my particular problem, NM actually delivers the best results.

[image] stevengj: do you mean that it converges in 100x more iterations, or that each iteration is 100x slower?) I think we should go back to this question, because it seems very likely that someone translating a code from Fortran might have done some Fortran style things that would absolutel…

Without an example from Op it is pure guesswork.

Cool, I had not been able to find implementations with bsd or mit licenses before

[image] PRIMA: a package for solving general nonlinear optimization problems without using derivatives Optimization (Mathematical) Hi everyone, I am very glad to announce PRIMA , a package for solving general nonlinear optimization problems without using derivatives. PRIM…

Why Nelder-Mead minimization without minimal property check?

Specific Domains Optimization (Mathematical)

pnavaro October 19, 2023, 7:10am 18

Topic		Replies	Views
Comparison of derivative-free / black-box optimization algorithms in Julia Optimization (Mathematical) question , comparison	12	3233	February 14, 2024
How to put bounds on Nelder-Mead in Optim? Optimization (Mathematical) question	25	8075	February 2, 2024
Nelder Mead requires derivatives? Optimization (Mathematical) optimization	6	266	December 11, 2024
`Optimization.LBFGS` fails to converge while `Optim.NelderMead()` works General Usage question , optim , optimization	11	421	March 29, 2025
Convergence behavior in MATLAB vs Julia New to Julia question , matlab , optimization	12	1157	January 25, 2024

Why Nelder-Mead minimization without minimal property check?

Related topics