# About JuMP's speed of building an expression

**URL:** <https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018>\
**Category:** Optimization (Mathematical)\
**Tags:** question, jump\
**Created:** [May 15, 2025, 4:49am UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018 "2025-05-15T04:49:18Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![WalterMadelim](https://avatars.discourse-cdn.com/v4/letter/w/3e96dc/32.png) [@WalterMadelim](https://discourse.julialang.org/u/WalterMadelim)\
**Post date:** [May 15, 2025, 4:49am UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/1 "2025-05-15T04:49:18Z")

</div>

I find that it seems to be not that fast for JuMP to build a “large” expression  
I wonder: is this speed normal? Why? Can (or how can) it be improved somehow?

```julia
import LinearAlgebra.dot as dot
using JuMP, BenchmarkTools
N = 1000
m = JuMP.Model();
JuMP.@variable(m, x[1:N])
JuMP.@variable(m, y[1:N])
A = rand(N, N)
z, w = rand(N), rand(N)
@btime dot(z, A, w) # 252.100 μs (1 allocation: 16 bytes)
@btime ret = JuMP.@expression(m, dot(x, A, y)) # 45.682 s (22435129 allocations: 105.56 GiB)

```

Note the performance indices in the comment of the last line.  
(Additionally, we can make another test in which `A` is sparse, [here](https://discourse.julialang.org/t/gurobi12-reports-obj-value-obj-bound-in-a-min-program/126849/18) is a readily available code snippet)

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [May 15, 2025, 5:21am UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/2 "2025-05-15T05:21:00Z")

</div>

We’re missing a method for the three-argument `LinearAlgebra.dot`, so it falls back to the generic version which allocates a lot.

```Julia
julia> using JuMP

julia> import LinearAlgebra

julia> N = 300
300

julia> model = Model();

julia> @variable(model, x[1:N]);

julia> @variable(model, y[1:N]);

julia> A = rand(N, N);

julia> @time @expression(model, dot(x, A, y));
  1.174037 seconds (1.28 M allocations: 2.980 GiB, 32.74% gc time)

julia> @time @expression(model, dot(x, A * y));
  0.036104 seconds (4.87 k allocations: 25.038 MiB)

julia> @time @expression(model, x' * A * y);
  0.054529 seconds (4.87 k allocations: 25.038 MiB, 35.70% gc time)

julia> @which dot(x, A, y)
dot(x::AbstractVector, A::AbstractMatrix, y::AbstractVector)
     @ LinearAlgebra ~/.julia/juliaup/julia-1.10.9+0.x64.apple.darwin14/share/julia/stdlib/v1.10/LinearAlgebra/src/generic.jl:923

```

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [May 15, 2025, 5:22am UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/3 "2025-05-15T05:22:39Z")

</div>

how hard would it be to add 3 arg dot? It seems like it would be useful.

---

<div class="post-metadata">

**Author:** ![WalterMadelim](https://avatars.discourse-cdn.com/v4/letter/w/3e96dc/32.png) [@WalterMadelim](https://discourse.julialang.org/u/WalterMadelim)\
**Post date:** [May 15, 2025, 8:27am UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/4 "2025-05-15T08:27:23Z")

</div>

In LinearAlgebra, I believe that `x' * A * y` are different from `dot(x, A, y)`.  
The latter has a LAPACK specialized method. x-ref [my previous comment](https://discourse.julialang.org/t/inner-product-grammar-is-neater-than-sum-index-in-jump-modeling-but-triggers-warning/124528/29).

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [May 15, 2025, 9:17pm UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/5 "2025-05-15T21:17:05Z")

</div>

> In LinearAlgebra, I believe that `x' * A * y` are different from `dot(x, A, y)`

They’re equivalent for real inputs (which your JuMP code is):

```julia
julia> x = rand(10); A = rand(10, 10); y = rand(10);

julia> x' * A * y
7.631533368388598

julia> LinearAlgebra.dot(x, A, y)
7.631533368388598

```

---

<div class="post-metadata">

**Author:** ![WalterMadelim](https://avatars.discourse-cdn.com/v4/letter/w/3e96dc/32.png) [@WalterMadelim](https://discourse.julialang.org/u/WalterMadelim)\
**Post date:** [May 15, 2025, 11:50pm UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/6 "2025-05-15T23:50:06Z")

</div>

equivalent in results, but might not in terms of the code it traverses.  
Do you have a plan to add a method for 3-arg dot?  
I find that the style `y' * (A' * x)` might be optimal.

```julia
julia> @btime x' * A * y;
  52.078 ms (12077 allocations: 37.02 MiB)

julia> @btime y' * (A' * x);
  50.323 ms (12078 allocations: 37.02 MiB)

julia> @btime dot(x, A * y);
  51.340 ms (12076 allocations: 37.02 MiB)

julia> @btime dot(y, A' * x);
  51.997 ms (12077 allocations: 37.02 MiB)

julia> 

julia> @btime x' * A * y;
  52.874 ms (12077 allocations: 37.02 MiB)

julia> @btime y' * (A' * x);
  48.908 ms (12078 allocations: 37.02 MiB)

julia> @btime dot(x, A * y);
  49.874 ms (12076 allocations: 37.02 MiB)

julia> @btime dot(y, A' * x);
  52.799 ms (12077 allocations: 37.02 MiB)

```

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [May 15, 2025, 11:57pm UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/7 "2025-05-15T23:57:51Z")

</div>

> Do you have a plan to add a method for 3-arg dot?

Nope. It’s a bit of a pain to intercept the appropriate methods as the number of arguments grow without introducing method ambiguities.

I don’t think that `dot(x, A, y)` is a common operation for JuMP models, and I would encourage people to use the more readable `x' * A * y` (this does appear often, e.g., in `x' * Q * x` type models). What’s more, `x' * A * y` is fewer characters to type.

---

<div class="post-metadata">

**Author:** ![WalterMadelim](https://avatars.discourse-cdn.com/v4/letter/w/3e96dc/32.png) [@WalterMadelim](https://discourse.julialang.org/u/WalterMadelim)\
**Post date:** [May 16, 2025, 12:42am UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/8 "2025-05-16T00:42:39Z")

</div>

> [@odow](#):
>
> I don’t think that `dot(x, A, y)` is a common operation for JuMP models

I think it will be used for its being a bilinear term, which could be addressed by Gurobi.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [May 16, 2025, 12:59am UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/9 "2025-05-16T00:59:36Z")

</div>

I agree that `x' * A * y` is a somewhat common operation (but I assume, \<5% of JuMP users). My point is that I don’t think anyone (aside from perhaps, you 😄) knows, cares, or wants to write `LinearAlgebra.dot(x, A, y)` instead.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [May 16, 2025, 5:24am UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/10 "2025-05-16T05:24:06Z")

</div>

If I were doing quadratic programming and were an experienced Julia user, I’d probably write the objective like that right away: `dot(x, A, x) + dot(b, x) + c`

---

<div class="post-metadata">

**Author:** ![WalterMadelim](https://avatars.discourse-cdn.com/v4/letter/w/3e96dc/32.png) [@WalterMadelim](https://discourse.julialang.org/u/WalterMadelim)\
**Post date:** [May 16, 2025, 5:56am UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/11 "2025-05-16T05:56:55Z")

</div>

1. `B * x` is not a scalar
2. constant `c` don’t need to appear on the objective

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [May 16, 2025, 5:57am UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/12 "2025-05-16T05:57:30Z")

</div>

My bad, fixed the typo to `dot(b, x)`. And I am aware that a constant does not change a quadratic objective 😉

---

<div class="post-metadata">

**Author:** ![blegat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blegat/32/217090_2.png) [@blegat](https://discourse.julialang.org/u/blegat)\
**Post date:** [May 16, 2025, 6:15am UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/13 "2025-05-16T06:15:45Z")

</div>

@gdalle I agree it would be could be a common way users would write it. The issue is that we have a lot of methods to implement

```julia
julia> methods(LinearAlgebra.dot, (AbstractVector, AbstractMatrix, AbstractVector))
# 14 methods for generic function "dot" from LinearAlgebra:
  [1] dot(x::AbstractVector, transA::Transpose{<:Real}, y::AbstractVector)
     @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/LinearAlgebra/src/generic.jl:950
  [2] dot(x::AbstractVector, D::Diagonal, y::AbstractVector)
     @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/LinearAlgebra/src/diagonal.jl:960
  [3] dot(x::AbstractVector, adjA::Adjoint, y::AbstractVector)
     @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/LinearAlgebra/src/generic.jl:949
  [4] dot(x::AbstractVector, B::Bidiagonal, y::AbstractVector)
     @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/LinearAlgebra/src/bidiag.jl:800
  [5] dot(x::AbstractVector, A::UpperTriangular, y::AbstractVector)
     @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/LinearAlgebra/src/triangular.jl:706
  [6] dot(x::AbstractVector, A::Tridiagonal, y::AbstractVector)
     @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/LinearAlgebra/src/tridiag.jl:887
  [7] dot(x::AbstractVector, H::UpperHessenberg, y::AbstractVector)
     @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/LinearAlgebra/src/hessenberg.jl:340
  [8] dot(x::AbstractVector, A::UnitLowerTriangular, y::AbstractVector)
     @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/LinearAlgebra/src/triangular.jl:769
  [9] dot(x::AbstractVector, A::UnitUpperTriangular, y::AbstractVector)
     @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/LinearAlgebra/src/triangular.jl:727
 [10] dot(x::AbstractVector, A::LowerTriangular, y::AbstractVector)
     @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/LinearAlgebra/src/triangular.jl:749
 [11] dot(x::AbstractVector, S::SymTridiagonal, y::AbstractVector)
     @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/LinearAlgebra/src/tridiag.jl:227
 [12] dot(x::AbstractVector, A::Union{Hermitian{var"#s5029", var"#s5028"}, Symmetric{var"#s5029", var"#s5028"}, Symmetric{Complex{var"#s5029"}, var"#s5028"}} where {var"#s5029"<:Real, var"#s5028"<:Diagonal}, y::AbstractVector)
     @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/LinearAlgebra/src/special.jl:326
 [13] dot(x::AbstractVector, A::Union{Hermitian{T, S}, Hermitian{Complex{T}, S}, Symmetric{T, S}} where {T<:Real, S}, y::AbstractVector)
     @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/LinearAlgebra/src/symmetric.jl:550
 [14] dot(x::AbstractVector, A::AbstractMatrix, y::AbstractVector)
     @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/LinearAlgebra/src/generic.jl:931

```

Ideally, we should rewrite almost all of LinearAlgebra and SparseArrays in MutableArithmetics, make sure the relevant calls go towards our implementation via multiple dispatch whenever some of the element types are subtypes of `AbstractMutable` (this magic is done in the [dispatch.jl file](https://github.com/jump-dev/MutableArithmetics.jl/blob/master/src/dispatch.jl)), make sure it works on all Julia version we support without creating any method ambiguities. That’s even more work than just reimplementing LinearAlgebra and SparseArrays ^^  
In the long term, I think it would be best for these packages to have default implementations that work well with mutable types by already using `add_mul!!` so it would mean moving MutableArithmetics or part of it as dependency of these packages. Maybe now that these have been moved out of Julia, it can start being a possibility.

---

<div class="post-metadata">

**Author:** ![WalterMadelim](https://avatars.discourse-cdn.com/v4/letter/w/3e96dc/32.png) [@WalterMadelim](https://discourse.julialang.org/u/WalterMadelim)\
**Post date:** [May 16, 2025, 7:12am UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/14 "2025-05-16T07:12:53Z")

</div>

I didn’t get your full idea.  
The dispatch of `LinearAlgebra.jl` had better depends on `MutableArithmetics`, or the `MutableArithmetics` follows the dispatch pattern of `LinearAlgebra.jl`?

**PS** At least in terms of documentation/tutorials, JuMP is more agreeable.

---

<div class="post-metadata">

**Author:** ![blegat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blegat/32/217090_2.png) [@blegat](https://discourse.julialang.org/u/blegat)\
**Post date:** [May 16, 2025, 7:15am UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/15 "2025-05-16T07:15:58Z")

</div>

I meant that if `LinearAlgebra` was using `MutableArithmetics` internally it its default implementations then we wouldn’t need to touch at the dispatch part of `LinearAlgebra` and we could get rid of the `dispatch.jl` file of `MutableArithmetics`.

---

<div class="post-metadata">

**Author:** ![WalterMadelim](https://avatars.discourse-cdn.com/v4/letter/w/3e96dc/32.png) [@WalterMadelim](https://discourse.julialang.org/u/WalterMadelim)\
**Post date:** [May 16, 2025, 7:59am UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/16 "2025-05-16T07:59:51Z")

</div>

Great. I support.  
In a larger sense, Matrices in `LinearAlgebra.jl` and `SparseArrays.jl` should hold more objects than only `<:Real`. It may also hold `JuMP.VariableRef`, or `Symobolics.Num`, etc.  
The special structure is widely applicable, e.g. `TriDiagonal`, `Symmetric`, etc.  
Things can be generalized. Only for regular numerical arithmetics, it then dispatch to LAPACK, or SUITESPARSE, etc.

---

<div class="post-metadata">

**Author:** ![blegat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blegat/32/217090_2.png) [@blegat](https://discourse.julialang.org/u/blegat)\
**Post date:** [May 16, 2025, 8:15am UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/17 "2025-05-16T08:15:11Z")

</div>

Yes, they can hold anything. The only issue is that their default implementations use `+=` instead or a mutable `muladd`. See for instance:

> <https://github.com/JuliaLang/LinearAlgebra.jl/blob/8a1ecfb22f8c6cbfa783c8810ae0b475d0a110f2/src/matmul.jl#L603>

---

<div class="post-metadata">

**Author:** ![araujoms](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/araujoms/32/217734_2.png) [@araujoms](https://discourse.julialang.org/u/araujoms)\
**Post date:** [May 19, 2025, 12:50pm UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/18 "2025-05-19T12:50:55Z")

</div>

This would also significantly speed up linear algebra with `BigFloat`, no?

---

<div class="post-metadata">

**Author:** ![blegat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blegat/32/217090_2.png) [@blegat](https://discourse.julialang.org/u/blegat)\
**Post date:** [May 20, 2025, 4:44pm UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/19 "2025-05-20T16:44:29Z")

</div>

Yes, that would be a good argument in favor

---

<div class="post-metadata">

**Author:** ![araujoms](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/araujoms/32/217734_2.png) [@araujoms](https://discourse.julialang.org/u/araujoms)\
**Post date:** [May 21, 2025, 10:13am UTC](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018/20 "2025-05-21T10:13:30Z")

</div>

Perhaps you should suggest it at [LinearAlgebra](https://github.com/JuliaLang/LinearAlgebra.jl) then? I’d support it. Also, I’ve contributed a few functions to LinearAlgebra, I’d be happy to rewrite them in `MutableArithmetics`form.

[Next page](https://discourse.julialang.org/t/about-jumps-speed-of-building-an-expression/129018.md?page=2)