I do have a loss function that :

- Is expressed as \lVert \mathbf y - \mathbf A'\mathbf p(\mathbf x)\rVert_2^2, where \mathbf y is a simple vector, \mathbf A is a simple matrix (lower-triangular), and p_1,...,p_m are nasty polynomials.
- Polynomials p_1,...,p_m are computed through a recursive implementation, say a function
`p(m,x)`

that computes all of them at once and which should be considered a black box (the number of coefficients of the polynomials are in the millions if we try to express them explicitelyā¦).

Moreover:

- The loss is written in pure julia and automatic differentiation can pass through
- But the very polynomial nature of the loss gives it a huge amount of local minimas, and any gradient descent (e.g.
`Optim.LBFGS()`

) gets stuck on the nearest local minimum.

For all these reasons, I am currently optimizing through `Optim.ParticleSwarm()`

, which is a global algorithm for non-convex non-differentiable functions. It works well but itās quite slow.

Is there somewhere a global optimization algorithm that is more adapted to problems where:

- The loss function is clearly non-convex with a lot of local minima.
- But it is āsmoothā and automatic differentiation can pass through, and therefore local gradient descent are possibles.

It is required that the optimisation routines are implemented in pure julia as i use BigFloats. Furthermore if linear equality and bound contraints are posible itās a bonus