I suppose I am somewhat cynical, but arxiv is full of papers describing optimization algorithms that happen to be beat existing methods for a few specific test problems. I have implemented quite a few myself and found that it is worth being a bit conservative, especially if one cares about worst case scenarios in a broad class of nonlinear problems.
1 Like