# Is there a function compute matrix multiplication in a multimensional matrix?

**URL:** <https://discourse.julialang.org/t/is-there-a-function-compute-matrix-multiplication-in-a-multimensional-matrix/18106>\
**Category:** Performance\
**Tags:** question\
**Created:** [November 28, 2018, 5:46pm UTC](https://discourse.julialang.org/t/is-there-a-function-compute-matrix-multiplication-in-a-multimensional-matrix/18106 "2018-11-28T17:46:09Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dhananjay\_Ghangale](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dhananjay_ghangale/32/6249_2.png) [@Dhananjay\_Ghangale](https://discourse.julialang.org/u/Dhananjay_Ghangale)\
**Post date:** [November 28, 2018, 5:46pm UTC](https://discourse.julialang.org/t/is-there-a-function-compute-matrix-multiplication-in-a-multimensional-matrix/18106/1 "2018-11-28T17:46:09Z")

</div>

For matrices of n dimensions and same sizes is there a function that can compute matrix multiplication.

for example in matlab there exists a function call mtimesx ([MTIMESX - Fast Matrix Multiply with Multi-Dimensional Support - File Exchange - MATLAB Central](https://www.mathworks.com/matlabcentral/fileexchange/25977-mtimesx-fast-matrix-multiply-with-multi-dimensional-support)). it would be benificial if there could be something similar in Julia ? or if there is a way to call this mex funcion from jula?

---

<div class="post-metadata">

**Author:** ![simonbyrne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simonbyrne/32/19_2.png) [@simonbyrne](https://discourse.julialang.org/u/simonbyrne)\
**Post date:** [November 28, 2018, 6:08pm UTC](https://discourse.julialang.org/t/is-there-a-function-compute-matrix-multiplication-in-a-multimensional-matrix/18106/2 "2018-11-28T18:08:30Z")

</div>

For multiplication of higher-dimensional arrays it is typically easier to think in terms of tensors and tensor contractions: I would suggest looking at some of the tensor packages, e.g.:

- [https://github.com/KristofferC/Tensors.jl](https://github.com/KristofferC/Tensors.jl)
- [https://github.com/Jutho/TensorOperations.jl](https://github.com/Jutho/TensorOperations.jl)

---

<div class="post-metadata">

**Author:** ![antoine-levitt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/antoine-levitt/32/4008_2.png) [@antoine-levitt](https://discourse.julialang.org/u/antoine-levitt)\
**Post date:** [November 28, 2018, 6:21pm UTC](https://discourse.julialang.org/t/is-there-a-function-compute-matrix-multiplication-in-a-multimensional-matrix/18106/3 "2018-11-28T18:21:32Z")

</div>

If I understand correctly the doc ([https://github.com/nonkreon/mtimesx/blob/master/doc/mtimesx\_20110223.pdf](https://github.com/nonkreon/mtimesx/blob/master/doc/mtimesx_20110223.pdf)), the package claims to speedup matrix multiplication, but in practice it seems to be mostly useful in the case of complex matrix multiplication. Matlab uses a representation for complex arrays that store the real and imaginary parts separately (an array of N complex numbers is stored 2 arrays of N real numbers). This does not map cleanly to BLAS routines, and the native matrix multiplication routines are apparently suboptimal, which this package is supposed to address. Julia does not have this problem, having the (much saner) memory layout of storing the real and imaginary parts together, which maps to BLAS directly, and is already optimal (provided a good BLAS is used). There’s also some functionality for exploiting symmetry, which julia also does automatically (A’\*B dispatches to a single BLAS call)

---

<div class="post-metadata">

**Author:** ![simonbyrne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simonbyrne/32/19_2.png) [@simonbyrne](https://discourse.julialang.org/u/simonbyrne)\
**Post date:** [November 28, 2018, 7:20pm UTC](https://discourse.julialang.org/t/is-there-a-function-compute-matrix-multiplication-in-a-multimensional-matrix/18106/4 "2018-11-28T19:20:39Z")

</div>

> [@antoine-levitt](#):
>
> Matlab uses a representation for complex arrays that store the real and imaginary parts separately (an array of N complex numbers is stored 2 arrays of N real numbers).

Apparently that has [now changed](https://www.mathworks.com/help/matlab/matlab_external/matlab-support-for-interleaved-complex.html) in newer versions of Matlab.

Anyone know the history of why Matlab stored them as separate parts? It seems a very odd choice.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [November 28, 2018, 7:33pm UTC](https://discourse.julialang.org/t/is-there-a-function-compute-matrix-multiplication-in-a-multimensional-matrix/18106/5 "2018-11-28T19:33:18Z")

</div>

> [@simonbyrne](#):
>
> I would suggest looking at some of the tensor packages, e.g.:
> 
> - [https://github.com/KristofferC/Tensors.jl](https://github.com/KristofferC/Tensors.jl)

I should say that Tensors.jl provide computations with “tensors” as typically defined in physics and not tensors as another word for multidimensional arrays.

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [November 28, 2018, 9:00pm UTC](https://discourse.julialang.org/t/is-there-a-function-compute-matrix-multiplication-in-a-multimensional-matrix/18106/6 "2018-11-28T21:00:36Z")

</div>

As the author of [StructsOfArrays](https://github.com/JuliaArrays/StructsOfArrays.jl#benchmark), I imagine you don’t think it is totally unreasonable? The example shows it could sometimes provide performance benefits / improve vectorization. I’d be interested in your thoughts there. I dug up the package as an example of why it could be useful, only to see your name on it, meaning you know way better than I do.

Julia’s `missing` values are also basically handled that way (a pair of arrays).

Has anyone tried something like StructsOfArrays with dual numbers? If so – did that improve performance?

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [November 28, 2018, 9:19pm UTC](https://discourse.julialang.org/t/is-there-a-function-compute-matrix-multiplication-in-a-multimensional-matrix/18106/7 "2018-11-28T21:19:36Z")

</div>

Mixing up different “Simon”'s is common in the Julia community 😉

---

<div class="post-metadata">

**Author:** ![antoine-levitt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/antoine-levitt/32/4008_2.png) [@antoine-levitt](https://discourse.julialang.org/u/antoine-levitt)\
**Post date:** [November 29, 2018, 6:52am UTC](https://discourse.julialang.org/t/is-there-a-function-compute-matrix-multiplication-in-a-multimensional-matrix/18106/8 "2018-11-29T06:52:49Z")

</div>

I think the argument with struct of arrays is that it’s useful to keep close together in memory things that will be used together. If you have an array of structures, it’s likely you’ll want to work on the array of one component (to add it to another array or something), which requires a stride, and then you can’t SIMD easily for instance. With complex numbers, how often do you need to work on the real part of an array without the imaginary one? Most operations (indeed, any holomorphic operation, which is probably why you used complex numbers in the first place) work on both the real and complex part at the same time.

---

<div class="post-metadata">

**Author:** ![Dhananjay\_Ghangale](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dhananjay_ghangale/32/6249_2.png) [@Dhananjay\_Ghangale](https://discourse.julialang.org/u/Dhananjay_Ghangale)\
**Post date:** [December 3, 2018, 4:33pm UTC](https://discourse.julialang.org/t/is-there-a-function-compute-matrix-multiplication-in-a-multimensional-matrix/18106/9 "2018-12-03T16:33:27Z")

</div>

Thanks for the answers !