# Inner product grammar is neater than sum(index) in JuMP modeling, But triggers Warning?

**URL:** <https://discourse.julialang.org/t/inner-product-grammar-is-neater-than-sum-index-in-jump-modeling-but-triggers-warning/124528>\
**Category:** Optimization (Mathematical)\
**Tags:** jump\
**Created:** [January 8, 2025, 9:35am UTC](https://discourse.julialang.org/t/inner-product-grammar-is-neater-than-sum-index-in-jump-modeling-but-triggers-warning/124528 "2025-01-08T09:35:37Z")\
**Posts on this page:** 9\
**Page:** 2

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [January 10, 2025, 7:38pm UTC](https://discourse.julialang.org/t/inner-product-grammar-is-neater-than-sum-index-in-jump-modeling-but-triggers-warning/124528/21 "2025-01-10T19:38:27Z")

</div>

This is a “bug” in MutableArithmetics, where we don’t have support for `SubArray`. The biggest problem in Julia is that there are no formal interfaces, so things work until they don’t, and then you hit a MethodError.

I’ve opened an issue: [similar\_array\_type with SubArray · Issue #316 · jump-dev/MutableArithmetics.jl · GitHub](https://github.com/jump-dev/MutableArithmetics.jl/issues/316)

---

<div class="post-metadata">

**Author:** ![WalterMadelim](https://avatars.discourse-cdn.com/v4/letter/w/3e96dc/32.png) [@WalterMadelim](https://discourse.julialang.org/u/WalterMadelim)\
**Post date:** [January 13, 2025, 3:08am UTC](https://discourse.julialang.org/t/inner-product-grammar-is-neater-than-sum-index-in-jump-modeling-but-triggers-warning/124528/22 "2025-01-13T03:08:25Z")

</div>

I was insane💀, since I could have written:

```julia
sum(p .* M_vector)

```

or in my “another example”

```julia
sum(p .* eachslice(M_Array; dims = 3))

```

which were concise, although still a slight bit of inconvenient compared to

```julia
ip(p, M_vector)

```

But after all, the MethodError in MutableArithmetics is revealed💀.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [January 13, 2025, 3:13am UTC](https://discourse.julialang.org/t/inner-product-grammar-is-neater-than-sum-index-in-jump-modeling-but-triggers-warning/124528/23 "2025-01-13T03:13:43Z")

</div>

I’m fixing MA: [Fix similar\_array\_type for SubArray by odow · Pull Request #317 · jump-dev/MutableArithmetics.jl · GitHub](https://github.com/jump-dev/MutableArithmetics.jl/pull/317)

---

<div class="post-metadata">

**Author:** ![blegat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blegat/32/217090_2.png) [@blegat](https://discourse.julialang.org/u/blegat)\
**Post date:** [January 13, 2025, 11:02am UTC](https://discourse.julialang.org/t/inner-product-grammar-is-neater-than-sum-index-in-jump-modeling-but-triggers-warning/124528/24 "2025-01-13T11:02:43Z")

</div>

When I run `ip(p, M_vector)`, I get

```julia
ERROR: MethodError: no method matching zero(::Type{Matrix{AffExpr}})
The function `zero` exists, but no method is defined for this combination of argument types.

Closest candidates are:
  zero(::Type{Union{}}, Any...)
   @ Base number.jl:310
  zero(::Type{Dates.Time})
   @ Dates ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Dates/src/types.jl:460
  zero(::Type{Pkg.Resolve.VersionWeight})
   @ Pkg ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/Resolve/versionweights.jl:15
  ...

Stacktrace:
 [1] neutral_element(::typeof(+), T::Type)
   @ MutableArithmetics ~/.julia/dev/MutableArithmetics/src/reduce.jl:25
 [2] fused_map_reduce(::typeof(add_dot), ::Vector{VariableRef}, ::Vector{Matrix{Float64}})
   @ MutableArithmetics ~/.julia/dev/MutableArithmetics/src/reduce.jl:44
 [3] operate(::typeof(dot), x::Vector{VariableRef}, y::Vector{Matrix{Float64}})
   @ MutableArithmetics ~/.julia/dev/MutableArithmetics/src/implementations/LinearAlgebra.jl:508
 [4] dot(x::Vector{VariableRef}, y::Vector{Matrix{Float64}})
   @ MutableArithmetics ~/.julia/dev/MutableArithmetics/src/dispatch.jl:58

```

I have opened an issue [dot product fails when the result is a matrix · Issue #318 · jump-dev/MutableArithmetics.jl · GitHub](https://github.com/jump-dev/MutableArithmetics.jl/issues/318)

---

<div class="post-metadata">

**Author:** ![WalterMadelim](https://avatars.discourse-cdn.com/v4/letter/w/3e96dc/32.png) [@WalterMadelim](https://discourse.julialang.org/u/WalterMadelim)\
**Post date:** [January 13, 2025, 1:17pm UTC](https://discourse.julialang.org/t/inner-product-grammar-is-neater-than-sum-index-in-jump-modeling-but-triggers-warning/124528/25 "2025-01-13T13:17:48Z")

</div>

Does it mean that after this revision, I could write

```julia
transpose(p) * [rand(2, 2) for _ in 1:length(p)] # p::Vector{JuMP.VariableRef}

```

without seeing an red ERROR messag?  
This `transpose(p) * v` Grammar is also likeable, a bit better than `sum(p .* v)`

---

<div class="post-metadata">

**Author:** ![blegat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blegat/32/217090_2.png) [@blegat](https://discourse.julialang.org/u/blegat)\
**Post date:** [January 13, 2025, 1:39pm UTC](https://discourse.julialang.org/t/inner-product-grammar-is-neater-than-sum-index-in-jump-modeling-but-triggers-warning/124528/26 "2025-01-13T13:39:28Z")

</div>

> [@WalterMadelim](#):
>
> `transpose(p) * [rand(2, 2) for _ in 1:length(p)]`

Yes, `transpose(p) * v` redirects to `LinearAlgebra._dot_nonrecursive` which is then caught by MutableArithmetics because `eltype(p)` is a JuMP variable and so it’s then redirected to `MutableArithmetics.fused_map_reduce(MutableArithmetics.add_mul, p, v)` which is working now with [Fix fused\_map\_reduce when result is a matrix by blegat · Pull Request #319 · jump-dev/MutableArithmetics.jl · GitHub](https://github.com/jump-dev/MutableArithmetics.jl/pull/319)

---

<div class="post-metadata">

**Author:** ![blegat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blegat/32/217090_2.png) [@blegat](https://discourse.julialang.org/u/blegat)\
**Post date:** [January 14, 2025, 8:44am UTC](https://discourse.julialang.org/t/inner-product-grammar-is-neater-than-sum-index-in-jump-modeling-but-triggers-warning/124528/27 "2025-01-14T08:44:57Z")

</div>

We just released MutableArithmetics v1.6.1 with the fix

---

<div class="post-metadata">

**Author:** ![WalterMadelim](https://avatars.discourse-cdn.com/v4/letter/w/3e96dc/32.png) [@WalterMadelim](https://discourse.julialang.org/u/WalterMadelim)\
**Post date:** [March 17, 2025, 4:43am UTC](https://discourse.julialang.org/t/inner-product-grammar-is-neater-than-sum-index-in-jump-modeling-but-triggers-warning/124528/28 "2025-03-17T04:43:07Z")

</div>

I reviewed some math books and concluded that `ip` (inner product) is a false name which conveyed a misconception. A correction could be

```julia
function sm(p, x) return sum(p .* x) end

```

where `s` stands for `vector sum` and `m` stands for `scalar multiplication`.

The `sm` should be a more fundamental operation, which makes sense in, e.g.,  
integration theory (`p` represents a _probability measure_ while `x` represents a _measurable function_).

And some related useful formulas are:

1. `@assert sm(A, B) == tr(transpose(A) * B)`
2. `@assert transpose(z) * A * w == tr(A * w * transpose(z))`
3. `@assert tr(A * B) == tr(B * A)` as long as `size(A) == size(transpose(B))`
4. `@assert sm(c .* x, y) == sm(c, diag(x * y'))`

where, `tr` and `diag` are from module `LinearAlgebra`.  
From the 4th point we see that `sm` is indeed more versatile that `tr`. Therefore I think it’s obfuscating to define an inner product using `tr` (as it was done in some textbooks).

---

<div class="post-metadata">

**Author:** ![WalterMadelim](https://avatars.discourse-cdn.com/v4/letter/w/3e96dc/32.png) [@WalterMadelim](https://discourse.julialang.org/u/WalterMadelim)\
**Post date:** [May 6, 2025, 2:34am UTC](https://discourse.julialang.org/t/inner-product-grammar-is-neater-than-sum-index-in-jump-modeling-but-triggers-warning/124528/29 "2025-05-06T02:34:44Z")

</div>

Given the discussion [here](https://discourse.julialang.org/t/dot-product/128704/6). I think we are motivated to fathom whether there is a standard (or at least recommended) way to write JuMP code that attains the optimal performance. @odow

Some very common cases include

```julia
import JuMP
using LinearAlgebra
model = JuMP.Model();
JuMP.@variable(model, x[1:3]);
JuMP.@variable(model, X[1:3, 1:3]);

JuMP.@expression(model, lin_scalar_1, rand(3)' * x)
JuMP.@expression(model, lin_scalar_2, dot(rand(3), x))

JuMP.@expression(model, quad_scalar_1, x' * rand(3, 3) * x)
JuMP.@expression(model, quad_scalar_2, dot(x, rand(3, 3), x))

JuMP.@expression(model, matrix_lin_scalar_1, rand(3)' * X * rand(3))
JuMP.@expression(model, matrix_lin_scalar_2, dot(rand(3), X, rand(3)))

JuMP.@expression(model, mat_lin_scalar_1, sum(rand(3, 3) .* X))
JuMP.@expression(model, mat_lin_scalar_2, dot(rand(3, 3), X))

```

It appears that they will have _different_ dispatches in `MutableArithmetics`, thus possibly different performances.

For a reference, notice the word `Unlike` in the following docstring

```julia
help?> *

  *(A, B::AbstractMatrix, C)
  A * B * C * D

  If the last factor is a vector, or the first a transposed vector, then it is efficient to deal with      
  these first. In particular x' * B * y means (x' * B) * y for an ordinary column-major B::Matrix. Unlike  
  dot(x, B, y), this allocates an intermediate array.

```

[Previous page](https://discourse.julialang.org/t/inner-product-grammar-is-neater-than-sum-index-in-jump-modeling-but-triggers-warning/124528.md?page=1)