# \[ANN\] Nonconvex 2.1 - more algorithms, documentation and autodiff features

**URL:** <https://discourse.julialang.org/t/ann-nonconvex-2-1-more-algorithms-documentation-and-autodiff-features/84264>\
**Category:** Package Announcements\
**Tags:** optimization\
**Created:** [July 15, 2022, 11:54am UTC](https://discourse.julialang.org/t/ann-nonconvex-2-1-more-algorithms-documentation-and-autodiff-features/84264 "2022-07-15T11:54:07Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [July 15, 2022, 11:54am UTC](https://discourse.julialang.org/t/ann-nonconvex-2-1-more-algorithms-documentation-and-autodiff-features/84264/1 "2022-07-15T11:54:08Z")

</div>

I have been slowly adding features and documentation to [`Nonconvex.jl`](https://github.com/JuliaNonconvex/Nonconvex.jl) and the [`JuliaNonconvex`](https://github.com/JuliaNonconvex) organization at large and it’s time to make a new announcement. Here are some updates with the new version.

- The README and docs are now updated to include [a summary table of the algorithms available.](https://github.com/JuliaNonconvex/Nonconvex.jl#algorithms)

- The documentation was significantly improved, especially the [“Gradients, Jacobians and Hessians”](https://julianonconvex.github.io/Nonconvex.jl/stable/gradients/gradients/) and [“Algorithms”](https://julianonconvex.github.io/Nonconvex.jl/stable/algorithms/algorithms/) sections.

- [Custom gradients and hessians](https://julianonconvex.github.io/Nonconvex.jl/stable/gradients/user_defined/) got easier.

- You can use [symbolic differentiation for specific functions or the entire model.](https://julianonconvex.github.io/Nonconvex.jl/stable/gradients/symbolic/)

- You can use [alternative AD backends for specific functions or the entire model.](https://julianonconvex.github.io/Nonconvex.jl/stable/gradients/other_ad/)

- You can use [sparse forward mode AD for specific functions or the entire model.](https://julianonconvex.github.io/Nonconvex.jl/stable/gradients/sparse/)

- You can use [ChainRules frule with ForwardDiff.](https://julianonconvex.github.io/Nonconvex.jl/stable/gradients/chainrules_fd/)

- You can easily define [implicit functions](https://julianonconvex.github.io/Nonconvex.jl/stable/gradients/implicit/) to make use of the implicit function theorem.

- You can [track the history of specific functions and their gradients.](https://julianonconvex.github.io/Nonconvex.jl/stable/gradients/history/)

- NonconvexIpopt supports [symbolic and/or sparse first and second order differentiation of the Lagrangian function](https://julianonconvex.github.io/Nonconvex.jl/stable/algorithms/ipopt/). The same is true for `NonconvexPavito` and `NonconvexJuniper` which use `NonconvexIpopt` as a sub-solver.

- [Metaheuristics.jl](https://github.com/jmejia8/Metaheuristics.jl) is now wrapped and [documented.](https://julianonconvex.github.io/Nonconvex.jl/stable/algorithms/metaheuristics/)

- [NOMAD.jl](https://github.com/bbopt/NOMAD.jl) is now wrapped and [documented](https://julianonconvex.github.io/Nonconvex.jl/stable/algorithms/nomad/), allowing the customization of the algorithm to the model where some constraints can be declared explicit and others declared progressive.

- The TOBS algorithm (a nonlinear, binary, heuristic optimization algorithm from the field of topology optimization) has been [implemented](https://github.com/JuliaNonconvex/NonconvexTOBS.jl) by @LucasMSpereira and [documented](https://julianonconvex.github.io/Nonconvex.jl/stable/algorithms/tobs/).

Happy optimization!

---

<div class="post-metadata">

**Author:** ![cgeoga](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cgeoga/32/216186_2.png) [@cgeoga](https://discourse.julialang.org/u/cgeoga)\
**Post date:** [July 15, 2022, 1:41pm UTC](https://discourse.julialang.org/t/ann-nonconvex-2-1-more-algorithms-documentation-and-autodiff-features/84264/2 "2022-07-15T13:41:56Z")

</div>

Every time I look at the package and the documentation I see cool new features and learn something new. Thanks so much for your hard work in putting all of this together. I’ve been a raw `Ipopt.jl` user for a very long time, but this package is seriously tempting me. I love how light the dependency tree is, and it sure would cut down on boilerplate.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/224680_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [July 18, 2022, 5:17am UTC](https://discourse.julialang.org/t/ann-nonconvex-2-1-more-algorithms-documentation-and-autodiff-features/84264/3 "2022-07-18T05:17:56Z")

</div>

Great news, thanks!  
Are there any updates on integrating the `frule` conversion utility you developed into ForwardDiff? I thought this would be a very welcome feature but the [relevant issue](https://github.com/JuliaDiff/ForwardDiff.jl/issues/579) seems to be sleeping

---

<div class="post-metadata">

**Author:** ![Albert\_Zevelev](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/albert_zevelev/32/11844_2.png) [@Albert\_Zevelev](https://discourse.julialang.org/u/Albert_Zevelev)\
**Post date:** [July 18, 2022, 5:48am UTC](https://discourse.julialang.org/t/ann-nonconvex-2-1-more-algorithms-documentation-and-autodiff-features/84264/4 "2022-07-18T05:48:08Z")

</div>

Awesome.  
Would it help to create an optimization algorithm flow chart:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/4/5/45692509625c26961bb5ac310a88caef3ebfccfe.jpeg)

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [July 18, 2022, 7:54am UTC](https://discourse.julialang.org/t/ann-nonconvex-2-1-more-algorithms-documentation-and-autodiff-features/84264/5 "2022-07-18T07:54:45Z")

</div>

> [@gdalle](#):
>
> Are there any updates on integrating the `frule` conversion utility you developed into ForwardDiff? I thought this would be a very welcome feature but the [relevant issue](https://github.com/JuliaDiff/ForwardDiff.jl/issues/579) seems to be sleeping

No update. I think my workaround is not the most efficient implementation. It could use less copying by re-interpreting and using views of the array of duals. Also the feature is probably not under high enough demand to convince the ForwardDiff maintainers to add it but I hope people prove me wrong.

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [July 18, 2022, 8:00am UTC](https://discourse.julialang.org/t/ann-nonconvex-2-1-more-algorithms-documentation-and-autodiff-features/84264/6 "2022-07-18T08:00:56Z")

</div>

> [@Albert\_Zevelev](#):
>
> Would it help to create an optimization algorithm flow chart:

This can get too complicated too quickly. Each algorithm or family of algorithms has its pros and cons. Even in nonlinear programming only, some algorithms support nonlinear inequality constraints only (no equality constraints), some support only linear equality or manifold constraints, some support linear equality constraints and nonlinear inequality constraints, some support both nonlinear inequality and equality constraints, some support only convex constraints (with a nonlinear objective), etc.