# Slow matrix multiplication in Julia compared to Python numpy

**URL:** https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015
**Category:** New to Julia
**Tags:** question
**Created:** [May 19, 2018, 5:52pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015 "2018-05-19T17:52:54Z")
**Posts on this page:** 18
**Page:** 1

<div class="post-metadata">

### Author: ![lucagessi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lucagessi/32/4189_2.png) [@lucagessi](https://discourse.julialang.org/u/lucagessi)
#### Post date: [May 19, 2018, 5:52pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015/1 "2018-05-19T17:52:54Z")

</div>

Hello.  
I am starting using Julia. I like its sintax, simplicity and its claimed performances.  
I have been using Matlab/Octave a lot at university and now at work we are using Octave for licence reason. However Octave is really slow and here I have read about Julia.  
I have tried some simple matrix multiplication (A\*B, not element wise one) but it is really slow.  
Octave is faster and Python numpy too.  
I am using Julia version 0.6.2 on Ubuntu notebook for the test but the same problem occurs in Windows 10.  
Python code below takes 0.006 seconds:

```julia
import numpy
import cProfile

n = 1000;
x=numpy.random.random((n,n))
y=numpy.random.random((n,n))
cProfile.run("x*y")

```

Julia code takes 0.1 seconds.

```julia
n = 1000;
a = rand(n,n);
@time a*a;

```

I don’t understand why is so slow.  
Thank you

---

<div class="post-metadata">

### Author: ![bernhard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bernhard/32/2619_2.png) [@bernhard](https://discourse.julialang.org/u/bernhard)
#### Post date: [May 19, 2018, 6:08pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015/2 "2018-05-19T18:08:22Z")

</div>

How many threads is numpy using?  
How many threads is julia (or blas in julia) using?

---

<div class="post-metadata">

### Author: ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)
#### Post date: [May 19, 2018, 6:12pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015/3 "2018-05-19T18:12:48Z")

</div>

Did you run it twice?

```julia
@time a*a
@time a*a

```

The first run compiles.  
This shouldn’t make much of a difference here, because it’s just calling an external (already compiled) BLAS routine, and the wrapper should already be compiled in your Julia system image.

So, which BLAS are each linked to? If you have an Intel processor, you can build Julia with MKL.

Finally, `x*y` in Python is element wise, so of course it is much faster. Try dot or matmul.  
Edit:  
[https://docs.scipy.org/doc/numpy-1.14.2/user/quickstart.html#basic-operations](https://docs.scipy.org/doc/numpy-1.14.2/user/quickstart.html#basic-operations)

---

<div class="post-metadata">

### Author: ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [May 19, 2018, 6:16pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015/4 "2018-05-19T18:16:06Z")

</div>

Unless I’m mistaken, `x*y` is elementwise multiplication in Python. Timing things in global scope in Julia is not a good way to get an accurate estimate of realistic runtimes. The right way to do it is this:

```julia
julia> using BenchmarkTools

julia> a = rand(1000, 1000);

julia> @benchmark $a .* $a # elementwise
BenchmarkTools.Trial:
  memory estimate: 7.63 MiB
  allocs estimate: 2
  --------------
  minimum time: 2.545 ms (0.00% GC)
  median time: 3.824 ms (0.00% GC)
  mean time: 4.986 ms (26.48% GC)
  maximum time: 78.045 ms (95.63% GC)
  --------------
  samples: 999
  evals/sample: 1

julia> @benchmark $a * $a # matmul
BenchmarkTools.Trial:
  memory estimate: 7.63 MiB
  allocs estimate: 2
  --------------
  minimum time: 51.575 ms (0.00% GC)
  median time: 57.945 ms (0.00% GC)
  mean time: 60.352 ms (3.93% GC)
  maximum time: 142.708 ms (58.50% GC)
  --------------
  samples: 83
  evals/sample: 1

```

On the same system, this is what I see for the same operations in Python:

```python
>>> import numpy
>>> import cProfile
>>> a = numpy.random.random((1000, 1000))
>>> cProfile.run("a * a")
         2 function calls in 0.010 seconds

   Ordered by: standard name

   ncalls tottime percall cumtime percall filename:lineno(function)
        1 0.010 0.010 0.010 0.010 <string>:1(<module>)
        1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}

>>> cProfile.run("a.dot(a)")
         3 function calls in 0.055 seconds

   Ordered by: standard name

   ncalls tottime percall cumtime percall filename:lineno(function)
        1 0.001 0.001 0.055 0.055 <string>:1(<module>)
        1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1 0.054 0.054 0.054 0.054 {method 'dot' of 'numpy.ndarray' objects}

```

So matrix multiplication seems to be the same in Julia and NumPy whereas Julia is significantly faster (4x) at elementwise multiplication. I’m not sure why, perhaps the operation is cheap enough that the overhead of calling out to C is significant in Python (and doesn’t exist in Julia). Edit: no, I tried a 10,000^2 matrix and the relative performance didn’t improve much—3x instead of 4x.

---

<div class="post-metadata">

### Author: ![lucagessi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lucagessi/32/4189_2.png) [@lucagessi](https://discourse.julialang.org/u/lucagessi)
#### Post date: [May 19, 2018, 6:16pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015/5 "2018-05-19T18:16:20Z")

</div>

I don’t know how to see that. I mean read number of thread.

I tried _ccall((:openblas\_get\_num\_threads64_, Base.libblas\_name), Cint, ())\_ in Julia and returns 4.  
In numpy I don’t know how to read it.

---

<div class="post-metadata">

### Author: ![lucagessi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lucagessi/32/4189_2.png) [@lucagessi](https://discourse.julialang.org/u/lucagessi)
#### Post date: [May 19, 2018, 6:56pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015/6 "2018-05-19T18:56:36Z")

</div>

I have tried with a 10000 x 10000 matrix.  
In my pc:  
Numpy: 0.673 seconds  
Julia: too much. I am still waiting

---

<div class="post-metadata">

### Author: ![jlapeyre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jlapeyre/32/4514_2.png) [@jlapeyre](https://discourse.julialang.org/u/jlapeyre)
#### Post date: [May 19, 2018, 7:03pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015/7 "2018-05-19T19:03:49Z")

</div>

Exactly what did you try ? Did you get past the confusion between matrix and elementwise multiplication ?

---

<div class="post-metadata">

### Author: ![jlapeyre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jlapeyre/32/4514_2.png) [@jlapeyre](https://discourse.julialang.org/u/jlapeyre)
#### Post date: [May 19, 2018, 7:12pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015/8 "2018-05-19T19:12:52Z")

</div>

On my machine, both python and Julia are using eight threads. I find that both elementwise multiplication and matrix multiplication take the same amount of time for the two languages.

```python
In [1]: import numpy

In [2]: import cProfile

In [3]: n = 10000;

In [4]: x=numpy.random.random((n,n))
   ...: y=numpy.random.random((n,n))
   ...: 

In [5]: cProfile.run("x*y")
         2 function calls in 0.391 seconds

   Ordered by: standard name

   ncalls tottime percall cumtime percall filename:lineno(function)
        1 0.391 0.391 0.391 0.391 <string>:1(<module>)
        1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}

In [6]: cProfile.run("x.dot(y)")
         3 function calls in 22.306 seconds

   Ordered by: standard name

   ncalls tottime percall cumtime percall filename:lineno(function)
        1 0.056 0.056 22.306 22.306 <string>:1(<module>)
        1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1 22.250 22.250 22.250 22.250 {method 'dot' of 'numpy.ndarray' objects}

```

```julia
julia> n = 10000;

julia> x = rand(n,n);

julia> y = rand(n,n);

julia> x .* y;

julia> @time x .* y;
  0.365343 seconds (30 allocations: 762.941 MiB, 10.40% gc time)

julia> @time x * y;
 23.990639 seconds (6 allocations: 762.940 MiB, 0.12% gc time)

```

EDIT: I committed crimes by not doing a proper benchmark and by timing in the top-level. But, in this case it doesn’t make much difference.

---

<div class="post-metadata">

### Author: ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [May 19, 2018, 7:14pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015/9 "2018-05-19T19:14:07Z")

</div>

Are you still comparing elementwise multiplication in Python with matrix multiplication in Julia? Because a 10x increase in `n` is expected to be a 100x slowdown in elementwise multiply, which matches the time you’re reporting for NumPy—about 0.6 seconds. Matrix multiply is super-linear in the size of the matrix, so you would expect a much bigger slowdown for that, which is exactly what you’re seeing in Julia.

---

<div class="post-metadata">

### Author: ![davidbp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidbp/32/463_2.png) [@davidbp](https://discourse.julialang.org/u/davidbp)
#### Post date: [May 19, 2018, 7:47pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015/10 "2018-05-19T19:47:31Z")

</div>

> [@lucagessi](#):
>
> ```julia
> @time a*a;
> 
> ```

That is indeed elementwise product in Python.  
Matrix multiply in python should be ` a @ a` or `np.matmul(a,a)`.

---

<div class="post-metadata">

### Author: ![lucagessi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lucagessi/32/4189_2.png) [@lucagessi](https://discourse.julialang.org/u/lucagessi)
#### Post date: [May 19, 2018, 8:45pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015/11 "2018-05-19T20:45:00Z")

</div>

Are you sure a\*a is element wise?  
From documentation:  
[https://docs.julialang.org/en/release-0.6/stdlib/linalg/](https://docs.julialang.org/en/release-0.6/stdlib/linalg/)

---

<div class="post-metadata">

### Author: ![lucagessi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lucagessi/32/4189_2.png) [@lucagessi](https://discourse.julialang.org/u/lucagessi)
#### Post date: [May 19, 2018, 8:48pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015/12 "2018-05-19T20:48:03Z")

</div>

I want to understand why is so slow. At the moment I haven’t a precise application

---

<div class="post-metadata">

### Author: ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [May 19, 2018, 8:48pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015/13 "2018-05-19T20:48:52Z")

</div>

No, `a*a` is matrix multiply in Julia but it’s elementwise multiply in Python, which is what you’re timing: `cProfile.run("x*y")`. That is why Python is faster and scales linearly with the size of the matrix.

---

<div class="post-metadata">

### Author: ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)
#### Post date: [May 19, 2018, 8:51pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015/14 "2018-05-19T20:51:37Z")

</div>

Python elementwise.

Julia matrix multiplication.

---

<div class="post-metadata">

### Author: ![lucagessi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lucagessi/32/4189_2.png) [@lucagessi](https://discourse.julialang.org/u/lucagessi)
#### Post date: [May 19, 2018, 8:54pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015/15 "2018-05-19T20:54:07Z")

</div>

Are you sure?  
I tried:

```julia
import numpy as np
x=np.matrix('1 2 3');
y=np.matrix('1;2;3');
x*y

```

And it results 14.  
I think is matrix multiplication.

---

<div class="post-metadata">

### Author: ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [May 19, 2018, 9:02pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015/16 "2018-05-19T21:02:15Z")

</div>

That’s a different type than `numpy.random.random` gives you and they do not behave the same way when you operate on them with `*`:

```python
>>> type(numpy.random.random((3,3)))
<type 'numpy.ndarray'>

>>> a = numpy.ones((3, 3))

>>> type(a)
<type 'numpy.ndarray'>

>>> a * a
array([[1., 1., 1.],
       [1., 1., 1.],
       [1., 1., 1.]])

>>> a.dot(a)
array([[3., 3., 3.],
       [3., 3., 3.],
       [3., 3., 3.]])

>>> m = numpy.matrix(a)

>>> type(m)
<class 'numpy.matrixlib.defmatrix.matrix'>

>>> m * m
matrix([[3., 3., 3.],
        [3., 3., 3.],
        [3., 3., 3.]])

>>> m.dot(m)
matrix([[3., 3., 3.],
        [3., 3., 3.],
        [3., 3., 3.]])

```

If you find it immensely confusing that `*` does completely different and incompatible things in Python/NumPy for subtly different array types, I can sympathize.

---

<div class="post-metadata">

### Author: ![lucagessi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lucagessi/32/4189_2.png) [@lucagessi](https://discourse.julialang.org/u/lucagessi)
#### Post date: [May 19, 2018, 9:12pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015/17 "2018-05-19T21:12:33Z")

</div>

Ok. I understand now 😃  
Thank you a lot.  
So I’ll wait for version 1 of Julia while using 0.6 😛

Good work!

---

<div class="post-metadata">

### Author: ![Seif\_Shebl](https://avatars.discourse-cdn.com/v4/letter/s/eada6e/32.png) [@Seif\_Shebl](https://discourse.julialang.org/u/Seif_Shebl)
#### Post date: [May 19, 2018, 9:28pm UTC](https://discourse.julialang.org/t/slow-matrix-multiplication-in-julia-compared-to-python-numpy/11015/18 "2018-05-19T21:28:19Z")

</div>

The fastest Python on the planet (Intel Python):

```julia
import numpy as np
import cProfile
import time

n = 10000;
x = np.random.random((n,n))
y = np.random.random((n,n))

t0 = time.clock()
z = np.matrix(x) * np.matrix(y)
t1 = time.clock()

total = t1-t0
print("Total (sec) =", total)

```

Which times:

```julia
Total (sec) = 10.513941910715321
[Finished in 12.4s]

```

And Julia (Julia Pro 0.6.0 for fair comparison):

```julia
julia> x = rand(10000,10000);

julia> y = rand(10000,10000);

julia> @time x*y;
  9.327905 seconds (6 allocations: 762.940 MiB, 0.12% gc time)

```

Hope the comparison is clear now! Happy Julia-ing, BTW.
