# Help speeding up problem with Convex.jl

**URL:** <https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005>\
**Category:** Optimization (Mathematical)\
**Tags:** question, convex-optimization\
**Created:** [April 4, 2022, 1:31pm UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005 "2022-04-04T13:31:27Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![josuagrw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/josuagrw/32/1015_2.png) [@josuagrw](https://discourse.julialang.org/u/josuagrw)\
**Post date:** [April 4, 2022, 1:31pm UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/1 "2022-04-04T13:31:28Z")

</div>

I’m currently solving a convex problem with Convex.jl and COSMO.jl. The text output from COSMO indicates that the solution is actually quite fast (less than a second) but it takes ca. 30 seconds to set the problem up with all constraints and the cost function.

All constraints are equalities which each link 3 components of a 1000-component `Variable` together (i.e. constraining 3D subvectors of the `Variable` onto various planes).

I’m currently adding the constraints individually with `add_constraint!` — am I doing something wrong or could I do this in a way which is easier for Convex.jl to set up? Should I switch away from COSMO.jl?

---

<div class="post-metadata">

**Author:** ![josuagrw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/josuagrw/32/1015_2.png) [@josuagrw](https://discourse.julialang.org/u/josuagrw)\
**Post date:** [April 4, 2022, 1:34pm UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/2 "2022-04-04T13:34:19Z")

</div>

ping @ericphanson @migarstka

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [April 4, 2022, 6:24pm UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/3 "2022-04-04T18:24:45Z")

</div>

It’s hard to offer advice without a reproducible example. Can you share the code?

---

<div class="post-metadata">

**Author:** ![josuagrw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/josuagrw/32/1015_2.png) [@josuagrw](https://discourse.julialang.org/u/josuagrw)\
**Post date:** [April 4, 2022, 7:17pm UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/4 "2022-04-04T19:17:54Z")

</div>

No, it’s much to complicated…

I suspect what is taking so much time is extracting the Hessian from the objective function.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [April 4, 2022, 7:28pm UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/5 "2022-04-04T19:28:24Z")

</div>

I don’t think we can offer advice without the code. There could be any number of reasons.

Have you tried JuMP?

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [April 4, 2022, 8:00pm UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/6 "2022-04-04T20:00:12Z")

</div>

Convex works best with vector/matrix constraints; scalar ones are slower for problem formulation (since the internals are not type stable). JuMP works on a scalar level so it could be a better fit as @odow says. The other main difference between the two is that Convex does automatic reformulations and checks that the problem is indeed convex, but you can do those reformulations by hand in JuMP if you need them. (Another difference is Convex supports high precision numeric types but that doesn’t seem relevant here).

---

<div class="post-metadata">

**Author:** ![lrnv](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lrnv/32/19373_2.png) [@lrnv](https://discourse.julialang.org/u/lrnv)\
**Post date:** [April 4, 2022, 8:56pm UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/7 "2022-04-04T20:56:37Z")

</div>

> [@josuagrw](#):
>
> I’m currently adding the constraints individually with `add_constraint!`

I suspect that if you give them all at once with a matrix-vector formulation it might be much faster

---

<div class="post-metadata">

**Author:** ![josuagrw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/josuagrw/32/1015_2.png) [@josuagrw](https://discourse.julialang.org/u/josuagrw)\
**Post date:** [April 4, 2022, 9:18pm UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/8 "2022-04-04T21:18:26Z")

</div>

I couldnt find anything about that in the Convex.jl documentation—how do I add constraints in matrix form?

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [April 4, 2022, 9:50pm UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/9 "2022-04-04T21:50:13Z")

</div>

For example, the constraints in [Entropy Maximization · Convex.jl](https://jump.dev/Convex.jl/stable/examples/general_examples/max_entropy/)

---

<div class="post-metadata">

**Author:** ![josuagrw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/josuagrw/32/1015_2.png) [@josuagrw](https://discourse.julialang.org/u/josuagrw)\
**Post date:** [April 5, 2022, 9:51am UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/10 "2022-04-05T09:51:39Z")

</div>

Thanks, I managed to make that work nicely with a sparse constraint matrix.

The program still spends 12–13 seconds on `solve!()` while _COSMO_ reports less than 1 second spent solving the problem.

Is there anyway to find out/optimize what Convex.jl is doing in those missing 10 seconds?

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [April 5, 2022, 10:00am UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/11 "2022-04-05T10:00:51Z")

</div>

> **[GitHub - timholy/ProfileView.jl: Visualization of Julia profiling data](https://github.com/timholy/ProfileView.jl)**
>
> Visualization of Julia profiling data. Contribute to timholy/ProfileView.jl development by creating an account on GitHub.

---

<div class="post-metadata">

**Author:** ![josuagrw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/josuagrw/32/1015_2.png) [@josuagrw](https://discourse.julialang.org/u/josuagrw)\
**Post date:** [April 5, 2022, 10:24am UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/12 "2022-04-05T10:24:24Z")

</div>

Good point, so the source of the delay appears to be this sorting operation in MathOptInterface.jl

[https://github.com/jump-dev/MathOptInterface.jl/blob/f559840765587598255e647dc144db8b74566d21/src/Utilities/functions.jl#L764](https://github.com/jump-dev/MathOptInterface.jl/blob/f559840765587598255e647dc144db8b74566d21/src/Utilities/functions.jl#L764)

I wonder if it would be possible to skip this step (by pre-sorting or something)

---

<div class="post-metadata">

**Author:** ![josuagrw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/josuagrw/32/1015_2.png) [@josuagrw](https://discourse.julialang.org/u/josuagrw)\
**Post date:** [April 5, 2022, 10:51am UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/13 "2022-04-05T10:51:15Z")

</div>

Ok, so as far as I can tell, when I call `solve!` my constraints pass through a Rube-Goldberg-machine inside `MathOptInterface.jl` at the end of which `sort_and_compress!` is called.

In the middle a method of `add_constraint()` which is located inside a file called `universalfallback.jl` is called. So I suspect the reason it takes so long, is some sort of slow fallback algorithm.

---

<div class="post-metadata">

**Author:** ![josuagrw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/josuagrw/32/1015_2.png) [@josuagrw](https://discourse.julialang.org/u/josuagrw)\
**Post date:** [April 5, 2022, 1:03pm UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/14 "2022-04-05T13:03:09Z")

</div>

A different way to ask is: Are there any Optimizers for Convex.jl which don’t rely on the universalfallback implementation?

---

<div class="post-metadata">

**Author:** ![josuagrw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/josuagrw/32/1015_2.png) [@josuagrw](https://discourse.julialang.org/u/josuagrw)\
**Post date:** [April 5, 2022, 2:35pm UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/15 "2022-04-05T14:35:12Z")

</div>

It appears that I can avoid the sorting procedure if I arrange my constraints in the right order:

[https://github.com/jump-dev/MathOptInterface.jl/blob/ff1c6d3d9c0b83335f74be0ca04beae56ac13d7d/src/Utilities/functions.jl#L776](https://github.com/jump-dev/MathOptInterface.jl/blob/ff1c6d3d9c0b83335f74be0ca04beae56ac13d7d/src/Utilities/functions.jl#L776)

Is there any documentation about the ideal order of constraints?

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [April 5, 2022, 6:33pm UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/16 "2022-04-05T18:33:42Z")

</div>

The internals of MOI are complicated, which makes it hard to debug or offer suggestions without a reproducible example. Can you not simplify your problem somehow? Hard code the data or read it from a JSON file?

Sorting isn’t related to the order of constraints, but the terms within an affine or quadratic constraint.

---

<div class="post-metadata">

**Author:** ![josuagrw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/josuagrw/32/1015_2.png) [@josuagrw](https://discourse.julialang.org/u/josuagrw)\
**Post date:** [April 6, 2022, 8:50am UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/17 "2022-04-06T08:50:38Z")

</div>

You’ve hit the nail on the head: The objective function I want to find an optimal solution for, is quite complicated (almost 100 lines of code, _without_ the constraints). In addition a lot of input data. I’ve tried to create a MWE — but when I simplify the objective function, the delay also vanishes.

Your answer explains that. Apparently, the 10 seconds are spent on processing the call graph of the objective.

---

<div class="post-metadata">

**Author:** ![josuagrw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/josuagrw/32/1015_2.png) [@josuagrw](https://discourse.julialang.org/u/josuagrw)\
**Post date:** [April 6, 2022, 8:52am UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/18 "2022-04-06T08:52:19Z")

</div>

It seems the only feasable solution is deriving the Hessian by hand and inputting it directly into _COSMO.jl_. I thought I could avoid that somehow with _Convex.jl_ but waiting 10 seconds to perform a 1 second optimization is not feasable.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [April 6, 2022, 9:00am UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/19 "2022-04-06T09:00:34Z")

</div>

Try JuMP instead. Convex has to do a lot of processing around the objective to prove convexity etc.

---

<div class="post-metadata">

**Author:** ![josuagrw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/josuagrw/32/1015_2.png) [@josuagrw](https://discourse.julialang.org/u/josuagrw)\
**Post date:** [April 6, 2022, 9:50am UTC](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005/20 "2022-04-06T09:50:11Z")

</div>

That’s quite frustrating… I specifically chose _Convex.jl_ because I know the problem I’m solving is a convex optimization. If I have to reimplement, then it makes more sense to me to do it with _COSMO_ — because then I know I will get the required performance.

If I now re-implemented in _JuMP_ then who knows if it will be faster, or slower, or if it will work at all.

[Next page](https://discourse.julialang.org/t/help-speeding-up-problem-with-convex-jl/79005.md?page=2)