[ANN] Nonconvex 2.1 - more algorithms, documentation and autodiff features

[mohamed82008]

I have been slowly adding features and documentation to Nonconvex.jl and the JuliaNonconvex organization at large and it’s time to make a new announcement. Here are some updates with the new version.

• The README and docs are now updated to include a summary table of the algorithms available.

• The documentation was significantly improved, especially the “Gradients, Jacobians and Hessians” and “Algorithms” sections.

• Custom gradients and hessians got easier.

• You can use symbolic differentiation for specific functions or the entire model.

• You can use alternative AD backends for specific functions or the entire model.

• You can use sparse forward mode AD for specific functions or the entire model.

• You can use ChainRules frule with ForwardDiff.

• You can easily define implicit functions to make use of the implicit function theorem.

• You can track the history of specific functions and their gradients.

• NonconvexIpopt supports symbolic and/or sparse first and second order differentiation of the Lagrangian function. The same is true for NonconvexPavito and NonconvexJuniper which use NonconvexIpopt as a sub-solver.

• Metaheuristics.jl is now wrapped and documented.

• NOMAD.jl is now wrapped and documented, allowing the customization of the algorithm to the model where some constraints can be declared explicit and others declared progressive.

• The TOBS algorithm (a nonlinear, binary, heuristic optimization algorithm from the field of topology optimization) has been implemented by @LucasMSpereira and documented.

Happy optimization!

[cgeoga]

Every time I look at the package and the documentation I see cool new features and learn something new. Thanks so much for your hard work in putting all of this together. I’ve been a raw Ipopt.jl user for a very long time, but this package is seriously tempting me. I love how light the dependency tree is, and it sure would cut down on boilerplate.

[gdalle]

Great news, thanks!
Are there any updates on integrating the frule conversion utility you developed into ForwardDiff? I thought this would be a very welcome feature but the relevant issue seems to be sleeping

[Albert_Zevelev]

Awesome.
Would it help to create an optimization algorithm flow chart:

[mohamed82008]

@gdalle

Are there any updates on integrating the frule conversion utility you developed into ForwardDiff? I thought this would be a very welcome feature but the relevant issue seems to be sleeping

No update. I think my workaround is not the most efficient implementation. It could use less copying by re-interpreting and using views of the array of duals. Also the feature is probably not under high enough demand to convince the ForwardDiff maintainers to add it but I hope people prove me wrong.

[mohamed82008]

@Albert_Zevelev

Would it help to create an optimization algorithm flow chart:

This can get too complicated too quickly. Each algorithm or family of algorithms has its pros and cons. Even in nonlinear programming only, some algorithms support nonlinear inequality constraints only (no equality constraints), some support only linear equality or manifold constraints, some support linear equality constraints and nonlinear inequality constraints, some support both nonlinear inequality and equality constraints, some support only convex constraints (with a nonlinear objective), etc.