Please don’t take this the wrong way, but numerical optimization is a very well established field. Generations of researchers have studied these problems in detail, characterizing the convergence properties and trade-offs of various solvers. Improving on the current best algorithms may take more than some heuristic idea.
If you want a nice introduction, I would recommend
Nocedal, Jorge, and Stephen Wright. Numerical optimization . Springer, 2006.