# Which Optimization package for large-scale structured nonlinear problems

**URL:** <https://discourse.julialang.org/t/which-optimization-package-for-large-scale-structured-nonlinear-problems/54449>\
**Category:** Optimization (Mathematical)\
**Tags:** jump, optim, optimization, nlsolve, nonlinear\
**Created:** [February 2, 2021, 11:14am UTC](https://discourse.julialang.org/t/which-optimization-package-for-large-scale-structured-nonlinear-problems/54449 "2021-02-02T11:14:50Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![egvila](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/egvila/32/21626_2.png) [@egvila](https://discourse.julialang.org/u/egvila)\
**Post date:** [February 2, 2021, 11:14am UTC](https://discourse.julialang.org/t/which-optimization-package-for-large-scale-structured-nonlinear-problems/54449/1 "2021-02-02T11:14:50Z")

</div>

Hello everyone,

As the title says, I would like some insight on what Julia Optimization package would be most appropriate to efficiently solve nonlinear problems with potentially exploitable sparsity structures (e.g. block diagonal Hessian).

I have come across various packages:

- NLPModels.jl emphasises the capability for user-defined jac/hess structure;
- JuMP.jl, Optim.jl, NLOpt.jl all seem to support nonlinear problems, but I’m not sure what happens under the hood for exploiting sparsity.  
Perhaps someone with experience on this could help?

Many thanks in advance:)

---

<div class="post-metadata">

**Author:** ![nantonel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nantonel/32/2889_2.png) [@nantonel](https://discourse.julialang.org/u/nantonel)\
**Post date:** [February 2, 2021, 12:07pm UTC](https://discourse.julialang.org/t/which-optimization-package-for-large-scale-structured-nonlinear-problems/54449/2 "2021-02-02T12:07:19Z")

</div>

Perhaps [StructuredOptimization.jl](https://github.com/kul-forbes/StructuredOptimization.jl) could be interesting for you as well!

It relies on solvers that do not build Hessian directly (it uses for example L-BFGS) but it can handle problems with sparse cost functions such as LASSO.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [February 2, 2021, 7:31pm UTC](https://discourse.julialang.org/t/which-optimization-package-for-large-scale-structured-nonlinear-problems/54449/3 "2021-02-02T19:31:04Z")

</div>

A good starting point is to use JuMP with Ipopt and see if the performance is acceptable.

How large is “large-scale”?

---

<div class="post-metadata">

**Author:** ![egvila](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/egvila/32/21626_2.png) [@egvila](https://discourse.julialang.org/u/egvila)\
**Post date:** [February 3, 2021, 11:18am UTC](https://discourse.julialang.org/t/which-optimization-package-for-large-scale-structured-nonlinear-problems/54449/4 "2021-02-03T11:18:05Z")

</div>

Thank you for the tip. Large enough to highly benefit from sparsity exploitation on its Jacobian and Hessian, would JuMP allow for that?

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [February 3, 2021, 4:57pm UTC](https://discourse.julialang.org/t/which-optimization-package-for-large-scale-structured-nonlinear-problems/54449/5 "2021-02-03T16:57:55Z")

</div>

JuMP commutes the Jacobian and Hessian using sparse reverse-mode AD. Try it out and see if the performance is acceptable.

---

<div class="post-metadata">

**Author:** ![egvila](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/egvila/32/21626_2.png) [@egvila](https://discourse.julialang.org/u/egvila)\
**Post date:** [February 4, 2021, 11:55am UTC](https://discourse.julialang.org/t/which-optimization-package-for-large-scale-structured-nonlinear-problems/54449/6 "2021-02-04T11:55:31Z")

</div>

Do you know if this is also done in the case of user-defined functions?

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [February 4, 2021, 6:43pm UTC](https://discourse.julialang.org/t/which-optimization-package-for-large-scale-structured-nonlinear-problems/54449/7 "2021-02-04T18:43:52Z")

</div>

For user-defined functions, we disable all hessians, and compute the gradient with forward mode AD. Depending on the model, this can cause significant issues.

If at all possible, you should write out your constraints algebraically.