in addition to the good points mentioned already, following the surface in high dimensions is going to lead you to a local optimum. even things like LBFGS from the Optim package was blown out of the water by BlackBoxOptim using differential evolution strategies on a problem I was working on yesterday because the problem had all sorts of local minima
1 Like