# Getting Oriented

**URL:** <https://discourse.julialang.org/t/getting-oriented/56970>\
**Category:** Optimization (Mathematical)\
**Created:** [March 11, 2021, 7:33pm UTC](https://discourse.julialang.org/t/getting-oriented/56970 "2021-03-11T19:33:53Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ross\_Boylan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ross_boylan/32/9210_2.png) [@Ross\_Boylan](https://discourse.julialang.org/u/Ross_Boylan)\
**Post date:** [March 11, 2021, 7:33pm UTC](https://discourse.julialang.org/t/getting-oriented/56970/1 "2021-03-11T19:33:53Z")

</div>

The introduction to the optimization topic in discourse refers to [http://www.juliaopt.org/](http://www.juliaopt.org/), but that site declares itself to be dead and obsolete, while referencing [https://jump.dev/](https://jump.dev/). But the latter does not appear to be a home base for Julia optimizers in general, but only those involved in JuMP. It does have pointers to some other sites.

In particular `Optim`, which I have been using, doesn’t seem to be there.

So where’s the best place to get an overview of the alternatives?

I’ve been using `Optim` since it allows me to take advantage of auto-differentiation. But I have a new problem that adds a linear equality constraints to the old problem (smooth nonlinear optimization) and am wondering what the best way to approach that is.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [March 11, 2021, 7:51pm UTC](https://discourse.julialang.org/t/getting-oriented/56970/2 "2021-03-11T19:51:50Z")

</div>

The package list of that site has many alternatives: [Packages](https://www.juliaopt.org/packages/)

Probably that got lost when JuMP got more attention.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [March 11, 2021, 7:58pm UTC](https://discourse.julialang.org/t/getting-oriented/56970/3 "2021-03-11T19:58:35Z")

</div>

See this post for some of the background on the JuliaOpt/JuMP-dev split:

> [@JuMP 0.21 is released](https://discourse.julialang.org/t/jump-0-21-is-released/34808/8):
>
> Despite the name, JuliaOpt has not, in practice, been the organization for “all optimization packages in Julia” for a number of years now. See [JuliaNLSolvers · GitHub](https://github.com/JuliaNLSolvers), [https://github.com/JuliaSmoothOptimizers](https://github.com/JuliaSmoothOptimizers), and [https://github.com/kul-forbes](https://github.com/kul-forbes) among others. Moving to jump-dev is also correcting this misleading impression. Discoverability is an important issue, and we would like to keep an up-to-date list of optimization-related packages in a highly visible place. The list is currently at [Packa…](http://www.juliaopt.org/packages/)

We point to other options on jump.dev:

> **[JuMP](https://jump.dev/#see-also)**
>
> JuMP is a modeling language and supporting packages for mathematical optimization in Julia.

I will update the introduction for this category.

Here’s a JuMP NLP example: [https://jump.dev/JuMP.jl/stable/examples/mle/](https://jump.dev/JuMP.jl/stable/examples/mle/). Ipopt is an excellent solver for constrained (convex) nonlinear problems.

Edit: I updated the intro: [About the Optimization (Mathematical) category](https://discourse.julialang.org/t/about-the-optimization-mathematical-category/90)

---

<div class="post-metadata">

**Author:** ![thisrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thisrod/32/10642_2.png) [@thisrod](https://discourse.julialang.org/u/thisrod)\
**Post date:** [March 12, 2021, 1:54am UTC](https://discourse.julialang.org/t/getting-oriented/56970/4 "2021-03-12T01:54:25Z")

</div>

> [@Ross\_Boylan](#):
>
> I have a new problem that adds a linear equality constraints to the old problem

It sounds like those constraints will define a manifold. Can you use the [manifolds](https://julianlsolvers.github.io/Optim.jl/stable/#algo/manifolds/) feature of Optim, define the `project_tangent!` method to be an orthogonal projector, and define `retract!` in terms of the same projector? I suspect there is a simple way to implement that with QR factorization, but I’d have to think about the details.

---

<div class="post-metadata">

**Author:** ![Ross\_Boylan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ross_boylan/32/9210_2.png) [@Ross\_Boylan](https://discourse.julialang.org/u/Ross_Boylan)\
**Post date:** [March 12, 2021, 6:16pm UTC](https://discourse.julialang.org/t/getting-oriented/56970/5 "2021-03-12T18:16:42Z")

</div>

Thank you for the suggestion. I don’t think manifolds are quite what I need, which is just to pick the optimal value of a vector b subject to a constraint matrix C such that Cb=0.

If there are q constraints and p parameters then C is q x p and this can be turned into an unconstrained optimization of p-q parameters using a QR decomposition of C^T to move between the constrained and unconstrained spaces. Though it would be nice not to have to do it by hand, since I’d also need to apply the transform to the Hessian to get the covariance matrix–the optimization is a conditional maximum likelihood estimation.

The problem could also be approached with Lagrange multipliers, but that seems less desirable computationally.

---

<div class="post-metadata">

**Author:** ![Ross\_Boylan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ross_boylan/32/9210_2.png) [@Ross\_Boylan](https://discourse.julialang.org/u/Ross_Boylan)\
**Post date:** [March 12, 2021, 6:49pm UTC](https://discourse.julialang.org/t/getting-oriented/56970/6 "2021-03-12T18:49:24Z")

</div>

Thanks for the pointer to the MLE example in JuMP. That looks like what I’m trying to do, and the JuMP docs indicate it does auto-differentiation. I do need to provide a custom nonlinear objective function, but apparently that’s supported.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [March 12, 2021, 7:09pm UTC](https://discourse.julialang.org/t/getting-oriented/56970/7 "2021-03-12T19:09:11Z")

</div>

Yes, see here: [Nonlinear Modeling · JuMP](https://jump.dev/JuMP.jl/dev/manual/nlp/#User-defined-Functions)