Benchmarking Julia vs NumPy

New to Julia
1

Hello, I decided to make couple of benchmarks against NumPy out of curiosity.
Since I don’t know Julia it would be nice if someone could take a second look because based on what I measured it doesn’t look like Julia is faster especially in matrix multiplication and norm.

The results are here https://gist.github.com/tastyminerals/f6764ff472c8e0e80301ad36ed34bf69

Julia 1.4.0 benchmark code is here: https://gist.github.com/tastyminerals/86eb070f4ea28a1dda2578de88d0e9f2

Thank you!

2

Numpy benchmark code is here: Numpy benchmarks · GitHub

3

It matters a lot how these benchmarks are executed. Numpy uses a blas library with a certain number of threads, executing for instance matrix-matrix multiplication in parallel. If one does not start Julia with multiple threads, it is hobbled.

There are several threads on this forum where you will find more information. Search for numpy and benchmark.

1 Like
4

Thank you.
Both were using multiple threads.
However, NumPy used MKL as backend (Linux conda).
Julia used OpenBLAS (Linux).

1 Like
5

You are using non-constant globals.

https://docs.julialang.org/en/v1/manual/performance-tips/

1 Like
6

Also, test2() is doing matrix multiplication.

And anyway, as you seem to know, all the work is done by the library. If you run both with the same library, then you will be benchmarking your benchmarking technique…

FWIW, on my computer the difference seems much less than 10x for test4, is it really this different on your computer, or was there a copy-paste error?

julia> @btime $float_matrixA * $float_matrixC; # test4, MKL — 0.001_856 in your table
  946.013 μs (2 allocations: 1.91 MiB)

julia> @btime $float_matrixA * $float_matrixC; # test4, OpenBLAS — 0.019_880 in your table
  1.191 ms (2 allocations: 1.91 MiB)
1 Like
7

I see, thank you. Just retested with all globals prepended with const. The results improved a but not drastically (still slower than NumPy).

8

As @improbable22 said, you’re comparing different operations. This line: https://gist.github.com/tastyminerals/86eb070f4ea28a1dda2578de88d0e9f2#file-julia_bench-jl-L31 is labeled as “element-wise multiplication”, but you’re actually doing matrix multiplication and then comparing it with element-wise multiplication in numpy.

1 Like
9

Indeed! Corrected.
Sorry, didn’t understand your last sentence. I thought just @btime is enough for such simple tests.

10

If you want to use MKL with Julia, the easiest way is to install MKL.jl.

2 Likes