# JuMP.jl: Supplying gradient of objective and jacobian of constraints in Nonlinear Optimization

**URL:** <https://discourse.julialang.org/t/jump-jl-supplying-gradient-of-objective-and-jacobian-of-constraints-in-nonlinear-optimization/106940>\
**Category:** Optimization (Mathematical)\
**Tags:** question, jump\
**Created:** [November 30, 2023, 10:18am UTC](https://discourse.julialang.org/t/jump-jl-supplying-gradient-of-objective-and-jacobian-of-constraints-in-nonlinear-optimization/106940 "2023-11-30T10:18:26Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![Strange\_Xue](https://avatars.discourse-cdn.com/v4/letter/s/e9a140/32.png) [@Strange\_Xue](https://discourse.julialang.org/u/Strange_Xue)\
**Post date:** [November 30, 2023, 10:18am UTC](https://discourse.julialang.org/t/jump-jl-supplying-gradient-of-objective-and-jacobian-of-constraints-in-nonlinear-optimization/106940/1 "2023-11-30T10:18:26Z")

</div>

Hi, I am faced with a troublesome optimization problem. As shown below, the non-linear optimization problem is easy to model and solve via Ipopt with JuMP.jl as interface. However, the objective function and constraints are time-consuming to compute, even with the assistance of auto-differenciation embedded in JuMP.jl.

\text{min} \; f(x)\\ \text{s.t.}\; g(x) = a \\ \quad \; s(x) = b\\

However, the jacobian and hessian of constraints is very sparse. Someone has provided the analytical jacobian and hessian of contraints, and analytical gradient of objective function to Ipopt with MATLAB opti as interface, and the speed is super fast.

Now, I know the analytical form of objective gradient and the jacobian and hessian of constraints. I wonder how can I provide those analytical expressions to Ipopt solver in JuMP.jl to speed up, I didn’t find any useful JuMP.jl docs in regard to this respect.

I guess MathOptInterface.jl can make it, but it is low-level and not easy to use. Besides, a lack of examples stopped me.

---

<div class="post-metadata">

**Author:** ![A\_C](https://avatars.discourse-cdn.com/v4/letter/a/da6949/32.png) [@A\_C](https://discourse.julialang.org/u/A_C)\
**Post date:** [November 30, 2023, 2:28pm UTC](https://discourse.julialang.org/t/jump-jl-supplying-gradient-of-objective-and-jacobian-of-constraints-in-nonlinear-optimization/106940/2 "2023-11-30T14:28:28Z")

</div>

You can find an example in the tutorial section of manual [(User Defined Hessian)](https://jump.dev/JuMP.jl/stable/tutorials/nonlinear/user_defined_hessians/). There is some information in the [Nonlinear Modelling](https://jump.dev/JuMP.jl/stable/manual/nonlinear/) part of the manual section. Search for hessian on the page. However, there is a comment:

> Finally, the matrix is treated as dense, so the performance will be poor on functions with high-dimensional input.

---

<div class="post-metadata">

**Author:** ![abelsiqueira](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abelsiqueira/32/47269_2.png) [@abelsiqueira](https://discourse.julialang.org/u/abelsiqueira)\
**Post date:** [November 30, 2023, 3:57pm UTC](https://discourse.julialang.org/t/jump-jl-supplying-gradient-of-objective-and-jacobian-of-constraints-in-nonlinear-optimization/106940/3 "2023-11-30T15:57:33Z")

</div>

If you don’t have to use JuMP, you can specify your optimization problem using JuliaSmoothOptimizers, for instance, using ManualNLPModels.

Here is an example with the gradient: [How to create a model from the function and its derivatives](https://jso.dev/tutorials/create-a-manual-model/)  
And the constructor docs: [Reference · ManualNLPModels.jl](https://jso.dev/ManualNLPModels.jl/stable/reference/#ManualNLPModels.NLPModel)  
You can then use Ipopt through NLPModelsIpopt. See these tutorials:

- [How to solve a small optimization problem with Ipopt + NLPModels](https://jso.dev/tutorials/solve-an-optimization-problem-with-ipopt/)
- [NLPModelsIpopt.jl tutorial](https://jso.dev/tutorials/introduction-to-nlpmodelsipopt/)

The last link also shows a different way to specify an optimization problem that does not depend on ManualNLPModels.jl.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [November 30, 2023, 8:49pm UTC](https://discourse.julialang.org/t/jump-jl-supplying-gradient-of-objective-and-jacobian-of-constraints-in-nonlinear-optimization/106940/4 "2023-11-30T20:49:35Z")

</div>

if you have just have some Julia functions `f`, `g`, and `s` that you want to minimize, the JuMP is not the best tool for the job. See [Should you use JuMP? · JuMP](https://jump.dev/JuMP.jl/stable/should_i_use/#You-want-to-optimize-a-complicated-Julia-function)

If you can share an example people may have suggestions, otherwise NLPModels is a good choice, or perhaps even the C API to Ipopt: [GitHub - jump-dev/Ipopt.jl: Julia interface to the Ipopt nonlinear solver](https://github.com/jump-dev/Ipopt.jl/tree/master#c-api).

---

<div class="post-metadata">

**Author:** ![Strange\_Xue](https://avatars.discourse-cdn.com/v4/letter/s/e9a140/32.png) [@Strange\_Xue](https://discourse.julialang.org/u/Strange_Xue)\
**Post date:** [December 2, 2023, 7:45am UTC](https://discourse.julialang.org/t/jump-jl-supplying-gradient-of-objective-and-jacobian-of-constraints-in-nonlinear-optimization/106940/5 "2023-12-02T07:45:39Z")

</div>

@odow The model I want to replicate is the random coefficient Logit model, a classical demand estimation model in empirical industrial organization.

Just as I mentioned, the model will be solved faster if we provide the analytical jacobian and hessian of constraints to Ipopt solver. Here is the linke to the paper.  
[https://onlinelibrary.wiley.com/doi/abs/10.3982/ECTA8585](https://onlinelibrary.wiley.com/doi/abs/10.3982/ECTA8585)  
The author [Jean-Pierre Dubé](https://www.jp-dube.com/research/MPECcode.html) has provided [MATLAB code](https://www.jp-dube.com/research/MPECcode.html) to replicate results.

In Julia ecosystem, BLPDemand.jl has implemented the estimation algorithm but without providing the jacobian and hessian of constraints, so the speed might be limited.

I am thinking about translate these MATLAB codes into Julia. The key takeaway is to feed the spase structure and values of jacobian and hessian to Ipopt.

Until now, I guess NLPModelsIPopt.jl could be a good tool, thanks to @abelsiqueira . Here is the link to the docs. [Tutorial · NLPModelsIpopt.jl](https://jso.dev/NLPModelsIpopt.jl/stable/tutorial/#Manual-input) . C API of Ipopt.jl might work as well.

I will apppreciate any other advice in this respect.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [December 3, 2023, 7:22am UTC](https://discourse.julialang.org/t/jump-jl-supplying-gradient-of-objective-and-jacobian-of-constraints-in-nonlinear-optimization/106940/6 "2023-12-03T07:22:12Z")

</div>

> but without providing the jacobian and hessian of constraints

Note that JuMP automatically computes this information. You don’t need to provide it analytically. Using the low-level API might be slightly faster, but it’s easy to make a mistake, and it takes longer to code. If you have a JuMP model and it works, then I’d just use that.

---

<div class="post-metadata">

**Author:** ![cvanaret](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cvanaret/32/11594_2.png) [@cvanaret](https://discourse.julialang.org/u/cvanaret)\
**Post date:** [December 3, 2023, 5:38pm UTC](https://discourse.julialang.org/t/jump-jl-supplying-gradient-of-objective-and-jacobian-of-constraints-in-nonlinear-optimization/106940/7 "2023-12-03T17:38:49Z")

</div>

A short note: since the problem doesn’t have any inequality constraints, IPOPT (a barrier or interior-point method) doesn’t even use its barrier-related techniques. It boils down to a Lagrange-Newton method, and the differences between different Newton-based methods are Hessian regularization (problem convexification) and globalization techniques (line search vs trust-region method & what kind of merit function).

---

<div class="post-metadata">

**Author:** ![Strange\_Xue](https://avatars.discourse-cdn.com/v4/letter/s/e9a140/32.png) [@Strange\_Xue](https://discourse.julialang.org/u/Strange_Xue)\
**Post date:** [December 4, 2023, 2:21am UTC](https://discourse.julialang.org/t/jump-jl-supplying-gradient-of-objective-and-jacobian-of-constraints-in-nonlinear-optimization/106940/8 "2023-12-04T02:21:18Z")

</div>

> [@odow](#):
>
> Note that JuMP automatically computes this information

What if the jacobian and hessian of constraints are very sparse? Does JuMP detect it automatically?, and will claiming sparsity structure reduce computing time?

---

<div class="post-metadata">

**Author:** ![brianguenter](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brianguenter/32/29519_2.png) [@brianguenter](https://discourse.julialang.org/u/brianguenter)\
**Post date:** [December 4, 2023, 2:38am UTC](https://discourse.julialang.org/t/jump-jl-supplying-gradient-of-objective-and-jacobian-of-constraints-in-nonlinear-optimization/106940/9 "2023-12-04T02:38:48Z")

</div>

[FastDifferentiation.jl](https://github.com/brianguenter/FastDifferentiation.jl) will handle sparse jacobians and hessians. It has limitations on problem size so it may not work for your case. But if it does it generates very efficient derivatives.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [December 4, 2023, 2:43am UTC](https://discourse.julialang.org/t/jump-jl-supplying-gradient-of-objective-and-jacobian-of-constraints-in-nonlinear-optimization/106940/10 "2023-12-04T02:43:07Z")

</div>

> What if the jacobian and hessian of constraints are very sparse? Does JuMP detect it automatically?

Yes. JuMP automatically computes sparse Jacobians and Hessians.

---

<div class="post-metadata">

**Author:** ![Strange\_Xue](https://avatars.discourse-cdn.com/v4/letter/s/e9a140/32.png) [@Strange\_Xue](https://discourse.julialang.org/u/Strange_Xue)\
**Post date:** [December 4, 2023, 7:00am UTC](https://discourse.julialang.org/t/jump-jl-supplying-gradient-of-objective-and-jacobian-of-constraints-in-nonlinear-optimization/106940/11 "2023-12-04T07:00:11Z")

</div>

> [@odow](#):
>
> If you have a JuMP model and it works, then I’d just use that.

Yep, I implemented a JuMP model and it works, but the speed is slow. I am thinking about show my code to ask for additional perfomance advice.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [December 4, 2023, 7:56am UTC](https://discourse.julialang.org/t/jump-jl-supplying-gradient-of-objective-and-jacobian-of-constraints-in-nonlinear-optimization/106940/12 "2023-12-04T07:56:40Z")

</div>

Yes please show your code. There might be other improvements

---

<div class="post-metadata">

**Author:** ![Strange\_Xue](https://avatars.discourse-cdn.com/v4/letter/s/e9a140/32.png) [@Strange\_Xue](https://discourse.julialang.org/u/Strange_Xue)\
**Post date:** [April 15, 2024, 9:14pm UTC](https://discourse.julialang.org/t/jump-jl-supplying-gradient-of-objective-and-jacobian-of-constraints-in-nonlinear-optimization/106940/13 "2024-04-15T21:14:31Z")

</div>

Thanks, I show my code in a seperate question. Here is the link.

> [@Speed up JuMP.jl Estimation When Jacobian and Hessian of Constraints are Sparse](https://discourse.julialang.org/t/speed-up-jump-jl-estimation-when-jacobian-and-hessian-of-constraints-are-sparse/112981):
>
> Hi guys, in last questions (link is shown below), I asked how to supply gradient of objective and jacobian of constraints in nonlinear optimization. But @odow told me I dont have to do that because JuMP.jl will automatically calculate them and feed them to Ipopt.jl. However, my model still runs slowly. So I show my code to find any opportunities to speed up. Thanks for your help. # \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* # # MATHEMATICAL PROGRAM WITH EQUILIBRIUM CONSTRAIN…