# OpenBLAS is faster than Intel MKL on AMD Hardware (Ryzen)

**URL:** <https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033>\
**Category:** Performance\
**Tags:** blas, lapack\
**Created:** [December 28, 2017, 2:49am UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033 "2017-12-28T02:49:32Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![cscherrer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cscherrer/32/7631_2.png) [@cscherrer](https://discourse.julialang.org/u/cscherrer)\
**Post date:** [August 18, 2019, 10:08pm UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/21 "2019-08-18T22:08:23Z")

</div>

> [@Elrod](#):
>
> By default, I thought OpenBLAS was only build with 16 threads.

Oh, I didn’t know that. Is there an easy way to check?

> [@Elrod](#):
>
> I also think you need to `addprocs` or start Julia with `-j$(nproc)` to use `parallel` , since it disables multithreading. That is, with `parallel=true` , each child process will multiply matrices. Still, I also saw a roughly 2x hit on performance from that.

There’s a `-p` option when you start Julia, is that what you mean? I didn’t think I needed this, since I set the environment variable. Am I misunderstanding this?

The [docs](https://docs.julialang.org/en/v1.3-dev/stdlib/LinearAlgebra/#LinearAlgebra.peakflops) say,

> If the underlying BLAS is using multiple threads, higher flop rates are realized. The number of BLAS threads can be set with [`BLAS.set_num_threads(n)`](https://docs.julialang.org/en/v1.3-dev/stdlib/LinearAlgebra/#LinearAlgebra.BLAS.set_num_threads).
> 
> If the keyword argument `parallel` is set to `true` , `peakflops` is run in parallel on all the worker processors. The flop rate of the entire parallel computer is returned. When running in parallel, only 1 BLAS thread is used. The argument `n` still refers to the size of the problem that is solved on each processor.

It doesn’t make sense to me why `parallel=true` would force single-threaded BLAS, but ok 🙂

> [@Elrod](#):
>
> Was `2.2912593368633377e11` with or without the 50% increase?

That was without:

```julia
 julia> BLAS.set_num_threads(16)

julia> LinearAlgebra.peakflops(16000)
3.5702446000519916e11

julia> BLAS.set_num_threads(32)

julia> LinearAlgebra.peakflops(16000)
3.293593157654745e11

```

> [@pistacliffcho](#):
>
> I’m still trying to learn more in this area

Me too!

> [@pistacliffcho](#):
>
> I think you only have 16 _physical_ threads available, with 32 _logical_ threads. If all your threads are being heavily used until completion of the task, it’s my understanding that it’s better to set to the physical thread count, rather than the logical thread count.

Correct on the first point. On the second, is that a general rule? Didn’t realize that. I’ve heard of people getting the best results with n-1 threads so one could still watch the mouse, etc. But I forget whether n was physical or logical in this case.

Just saw the additional details you both gave, that makes it much  
clearer.

> [@Elrod](#):
>
> For a more fair comparison, the 2950X ought to be overclocked similarly

Good point, I haven’t even looked into overclocking

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [August 18, 2019, 10:33pm UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/22 "2019-08-18T22:33:57Z")

</div>

> Oh, I didn’t know that. Is there an easy way to check?

You can check the build:  
[https://github.com/JuliaLang/julia/blob/master/deps/blas.mk#L25](https://github.com/JuliaLang/julia/blob/master/deps/blas.mk#L25)  
If you don’t mind compiling Julia from source, you could also change the line for your OS.

> There’s a `-p` option when you start Julia, is that what you mean? I didn’t think I needed this, since I set the environment variable. Am I misunderstanding this?

Ah, yes, that is what I meant. Which environmental variable?  
Also, docs on Julia master:

```julia
help?> LinearAlgebra.peakflops
  LinearAlgebra.peakflops(n::Integer=2000; parallel::Bool=false)

  peakflops computes the peak flop rate of the computer by using double precision gemm!. By default, if no arguments are specified, it multiplies a matrix of size n x n, where n = 2000. If the underlying BLAS is using multiple threads, higher flop rates are realized. The number of BLAS threads can be set with BLAS.set_num_threads(n).

  If the keyword argument parallel is set to true, peakflops is run in parallel on all the worker processors. The flop rate of the entire parallel computer is returned. When running in parallel, only 1 BLAS thread is used. The argument n still refers to the size of the problem that is solved on each processor.

```

"When running in parallel, only 1 BLAS thread is used. " I imagine the worker processes are the ones you get from `-p` or `addprocs`.

> Good point, I haven’t even looked into overclocking

I just moderately increase the clock speed, boot, launch Julia and try

```julia
X = rand(10^4,10^4);
@time inv(X);
@time inv(X);
@time inv(X);
@time foreach(inv, (X for _ in 1:100));

```

If the overclock is bad, the computer will normally crash the instant you hit “enter” on the first `@time inv(X)`. If it doesn’t crash, watch the temperatures (I use `watch -n0.5 sensors`) and speed (`watch -n1 "cat /proc/cpuinfo | grep MHz"`) and see how high the temperatures get, and make sure it’s actually running at the specified speed. Sometimes it’ll heat up and crash or throttle.  
Make sure it’s at a temperature you’re comfortable with.

If things crash, you can try and increase voltage. You’d have to look up what is safe.  
Also, (obviously) better safe than sorry. It’s not worth risking the computer crashing while you’re actually using it.

I also set XMP settings for memory, so that the RAM actually runs at the advertised speeds. I haven’t bothered to overclock beyond that yet, but it’d be a similar process. Maybe through in some large dot products / vector sums that are definitely memory bottle necked.

Also, my single thread benchmarks are actually worse now. Stock, the boost speed changes based on the number of cores working. I was lazy, and set the same speed for working cores (inactive cores still run much slower to save power). So I don’t think a single core boosts quite as high.  
But when I’m actually doing real work, all the cores are busy, so it’s not something I worry too much about. It’d just be cool to see great times when running single threaded `@benchmark`s. 😉

---

<div class="post-metadata">

**Author:** ![RoyiAvital](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/royiavital/32/571_2.png) [@RoyiAvital](https://discourse.julialang.org/u/RoyiAvital)\
**Post date:** [October 7, 2019, 4:56am UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/23 "2019-10-07T04:56:55Z")

</div>

Found nice trick to maximize MKL performance on AMD CPU’s based on Agner Fog findings: [Anaconda on Windows with MKL Support](https://github.com/fo40225/Anaconda-Windows-AMD).

Pay attnetion to the compilation trick: [https://github.com/fo40225/Anaconda-Windows-AMD/blob/master/site.cfg](https://github.com/fo40225/Anaconda-Windows-AMD/blob/master/site.cfg). Just incorporating Agner Fog’s CPU Dispatch.

Should be interesting to see comparison with this trick applied.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [October 9, 2019, 7:02pm UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/24 "2019-10-09T19:02:16Z")

</div>

I’ve got to watch this space over the next year - I’ll want to put together a machine that is superfast at running Julia, that will also serve as a high powered gaming machine for my kids (has to be able to run the upcoming Microsoft Flight Simulator 2020 at high frame rates, at at least 4K), as well as allow me to play with the GPUs with Julia.

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [October 10, 2019, 4:09am UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/25 "2019-10-10T04:09:51Z")

</div>

How high is your budget? Does it include the HEDT line ups?

IIRC, you’ve done a lot of low level / explicit SIMD for text processing?  
Sounds like you’d know how much/if you’d benefit from avx512.  
Thanks to strong competition from AMD, the new Cascade Lake-X CPUs (with avx512) cost half as much as the previous generation. Eg, $1000 for the 18 core part, which if over clocked should be able to hit around 2 teraflops with `LinearAlgebra.peakflops(16000)`, like the much older 7980xe. The 10 core part is $600.

If you won’t benefit from avx512, the AMD parts will probably be better. The Ryzen 3900X is 12 cores for $500, and 3950X will be 16 for $750. I have no idea about the new threadrippers. These chips do better on most benchmarks than Cascade Lake in single core performance relative to clock speed.

Julia is good at generating avx512 code, but lots of software, even software that could theoretically make good use of it like OpenBLAS, do not. Based on reviews, I don’t think games benefit. You’d know about your string libraries.

I could run benchmark scripts if it doesn’t take much work to set up.

@Royi, that would be pretty cool if someone can get that to work, especially with the new 7nm parts.

---

<div class="post-metadata">

**Author:** ![RoyiAvital](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/royiavital/32/571_2.png) [@RoyiAvital](https://discourse.julialang.org/u/RoyiAvital)\
**Post date:** [October 10, 2019, 3:24pm UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/26 "2019-10-10T15:24:24Z")

</div>

> [@Elrod](#):
>
> @Royi, that would be pretty cool if someone can get that to work, especially with the new 7nm parts.

Do you mean the patch for MKL?  
Well, one need to build it form the `lib` files in with that. It works as you can see in the repository.

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [May 19, 2020, 9:45am UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/27 "2020-05-19T09:45:18Z")

</div>

Anyone got their hands on a Ryzen 3900X and have some experiences to share?  
I’m thinking about getting one and am pretty much wondering if it’s a good time to get one, or there is something up-and-coming that is worth holding out for?

The 3900X seems to perform very well on the typical processor-review tests, but am unsure if this conclusion extends to linalg as well. I has also been quite a couple of months since it was released so I’m not sure if it’s about to be superseded soon?

---

<div class="post-metadata">

**Author:** ![Modatu](https://avatars.discourse-cdn.com/v4/letter/m/4bbf92/32.png) [@Modatu](https://discourse.julialang.org/u/Modatu)\
**Post date:** [May 19, 2020, 10:31am UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/28 "2020-05-19T10:31:14Z")

</div>

I have a Threadripper 1950X not really comparable, but I am really happy with it.

As always, the next generation is around the corner. 🙂  
AMD is planning (as far as I am informed) to roll-out their Zen 3 chips this year, so if you can wait a little bit longer, these will probably be called something like 4…(X).

---

<div class="post-metadata">

**Author:** ![RoyiAvital](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/royiavital/32/571_2.png) [@RoyiAvital](https://discourse.julialang.org/u/RoyiAvital)\
**Post date:** [May 19, 2020, 2:06pm UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/29 "2020-05-19T14:06:38Z")

</div>

You haven’t said what are you comparing the `AMD 3900x` to.

But generally speaking, it will be faster than any Intel CPU with less cores which doesn’t have `AVX512` for Multi Threaded Linear Algebra.

It will be also a decent CPU for single threaded operations yet Intel has higher frequency CPU’s which will beat it on those tasks (Probably in the 3-10% range).

If you use it with MKL, remember to use the [Acceleration of Intel MKL on AMD Ryzen CPU’s](https://discourse.julialang.org/t/31287) trick. Then you’ll get the best BLAS performance out there for non `AVX512` 12 cores CPU.

---

<div class="post-metadata">

**Author:** ![davidbp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidbp/32/463_2.png) [@davidbp](https://discourse.julialang.org/u/davidbp)\
**Post date:** [May 19, 2020, 2:15pm UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/30 "2020-05-19T14:15:42Z")

</div>

> [@RoyiAvital](#):
>
> But generally speaking, it will be faster than any Intel CPU with less cores which doesn’t have `AVX512` for Multi Threaded Linear Algebra.

This is highly debatable. Most intel CPUs have 1/2 GHz of the speed when using AVX512. [A table here](https://lemire.me/blog/2018/09/07/avx-512-when-and-how-to-use-these-new-instructions/) shows some numbers. Note that this is skylake. I wonder if this happens with the latest 10 nm chips. I have some collegues who found out that their code was faster with AVX2 than using AVX512 precisely because of this.

---

<div class="post-metadata">

**Author:** ![jlapeyre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jlapeyre/32/4514_2.png) [@jlapeyre](https://discourse.julialang.org/u/jlapeyre)\
**Post date:** [May 19, 2020, 2:19pm UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/31 "2020-05-19T14:19:39Z")

</div>

> [@baggepinnen](#):
>
> Anyone got their hands on a Ryzen 3900X and have some experiences to share?

I have the “non-X” version in a Clevo laptop.

```julia
julia> BLAS.set_num_threads(Sys.CPU_THREADS >> 1);

julia> LinearAlgebra.peakflops(16000)
3.841950244780208e11

julia> versioninfo()
Julia Version 1.6.0-DEV.58
Commit cfd7f48330 (2020-05-17 17:35 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: AMD Ryzen 9 3900 12-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, znver1)

```

The system is running a desktop with some stuff running, but really a light load. Repeating, the number are about `3.6--3.9 E11` flops.

---

<div class="post-metadata">

**Author:** ![RoyiAvital](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/royiavital/32/571_2.png) [@RoyiAvital](https://discourse.julialang.org/u/RoyiAvital)\
**Post date:** [May 19, 2020, 2:26pm UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/32 "2020-05-19T14:26:11Z")

</div>

> [@davidbp](#):
>
> > [@RoyiAvital](#):
> >
> > But generally speaking, it will be faster than any Intel CPU with less cores which doesn’t have `AVX512` for Multi Threaded Linear Algebra.
> 
> This is highly debatable. Most intel CPUs have 1/2 GHz of the speed when using AVX512. [A table here](https://lemire.me/blog/2018/09/07/avx-512-when-and-how-to-use-these-new-instructions/) shows some numbers. Note that this is skylake. I wonder if this happens with the latest 10 nm chips.

Even if it reduces 1 [GHz] it doubles the performance (Throughput) assuming memory BW isn’t the limiting factor. So for 4 [GHz] CPU in `AVX256` you get performance of 6 [GHz] instead of the potential 8 [GHz]. Still it gives you a nice boost in performance (50% for those numbers).  
The problem with the clock throttling is in a mixed code of `AVX512` and single threaded code. In that case the reduction of clock will hurt performance to a point it doesn’t make sense to use `AVX512`.

But for Linear Algebra which is usually Multi Threaded oriented code, believe me `AVX512` is a great feature.

You can ask @Elrod :-). Or just see his great numbers when his code utilizes `AVX512` well. Intel MKL on `AVX512` CPU’s beats CPU’s without `AVX512` easily.

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [May 19, 2020, 2:26pm UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/33 "2020-05-19T14:26:16Z")

</div>

My newer CPU:

```julia
julia> using LinearAlgebra

julia> BLAS.set_num_threads(Sys.CPU_THREADS >> 1)

julia> LinearAlgebra.peakflops(16000)
2.1151237886809092e12

julia> LinearAlgebra.peakflops(16000)
2.1014127914913848e12

julia> LinearAlgebra.peakflops(16000)
2.1263672666765479e12

julia> versioninfo(verbose=true)
Julia Version 1.5.0-DEV.872
Commit 1345a043d2* (2020-05-06 02:30 UTC)
Platform Info:
  OS: Linux (x86_64-generic-linux)
  uname: Linux 5.6.11-948.native #1 SMP Wed May 6 00:04:43 PDT 2020 x86_64 unknown
  CPU: Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz:
                 speed user nice sys idle irq
       #1-36 3981 MHz 165301374 s 3337 s 797186 s 3161664577 s 376496 s

  Memory: 125.55207824707031 GB (98871.29296875 MB free)
  Uptime: 924561.0 sec
  Load Avg: 1.7197265625 0.60107421875 0.2099609375
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)

```

> [@davidbp](#):
>
> This is highly debatable. Most intel CPUs have 1/2 GHz of the speed when using AVX512.

“Intel Xeon Gold 5120” is not “Most intel CPUs”. For example, mine (above) runs 18 cores without AVX at 4.6 GHz, with AVX(2) at 4.3 GHz, and with AVX512 at 4.1GHz.

> [@RoyiAvital](#):
>
> You can ask @Elrod :-). Or just see his great numbers when his code utilizes `AVX512` well.

I love it. Obviously I put a lot of effort into making use of it (in particular, LoopVectorization), but it pays off.

---

<div class="post-metadata">

**Author:** ![davidbp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidbp/32/463_2.png) [@davidbp](https://discourse.julialang.org/u/davidbp)\
**Post date:** [May 19, 2020, 2:28pm UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/34 "2020-05-19T14:28:57Z")

</div>

That is a pretty “badass” CPU. CPUs without the “gold” adjective don’t have this behaviour. Please keep working on LoopVectorization it’s awesome!

---

<div class="post-metadata">

**Author:** ![mbaz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbaz/32/17295_2.png) [@mbaz](https://discourse.julialang.org/u/mbaz)\
**Post date:** [May 19, 2020, 2:54pm UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/35 "2020-05-19T14:54:23Z")

</div>

> [@davidbp](#):
>
> I wonder if this happens with the latest 10 nm chips.

If you tell me what to run, I’ll be happy to provide numbers from my Ice Lake i5.

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [May 19, 2020, 3:50pm UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/36 "2020-05-19T15:50:34Z")

</div>

Thanks all for sharing 🙂 Chris’ CPU is perhaps a bit pricey, but does of course seem extremely competent!  
I think I can live without AVX512, many workloads do not benefit from it at all, but do benefit from a high core count, something AMD offers for a very reasonable price while also offering quite competitive single-core performance. The fact that AMD is soon moving to a new socket with some new technologies still make me wonder if it’s time to hold out for a few more months though 😛

---

<div class="post-metadata">

**Author:** ![tamasgal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamasgal/32/27946_2.png) [@tamasgal](https://discourse.julialang.org/u/tamasgal)\
**Post date:** [May 19, 2020, 4:23pm UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/37 "2020-05-19T16:23:16Z")

</div>

Interesting, let me add an older Xeon just for the sake of completeness. This is my desktop:

```julia
julia> using LinearAlgebra

julia> BLAS.set_num_threads(Sys.CPU_THREADS >> 1);

julia> LinearAlgebra.peakflops(16000)
2.036555694312491e11

julia> versioninfo(verbose=true)
Julia Version 1.4.1
Commit 381693d3df* (2020-04-14 17:20 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      "Manjaro Linux"
  uname: Linux 4.14.172-1-MANJARO #1 SMP PREEMPT Fri Feb 28 21:28:00 UTC 2020 x86_64 unknown
  CPU: Intel(R) Xeon(R) CPU E3-1230 v6 @ 3.50GHz:
              speed user nice sys idle irq
       #1 3773 MHz 14956550 s 3586749 s 5883343 s 572677402 s 0 s
       #2 3774 MHz 16758671 s 3570021 s 5807040 s 571980878 s 0 s
       #3 3860 MHz 19823855 s 3551664 s 5926076 s 569028854 s 0 s
       #4 3746 MHz 24385969 s 3518101 s 5655991 s 565108513 s 0 s
       #5 3844 MHz 19898516 s 3536843 s 5572273 s 569492118 s 0 s
       #6 3779 MHz 16081160 s 3557193 s 5681533 s 572925080 s 0 s
       #7 3811 MHz 16224667 s 3569879 s 5676842 s 572804372 s 0 s
       #8 3803 MHz 14050354 s 3596319 s 5671032 s 575381836 s 0 s

  Memory: 31.276653289794922 GB (262.8671875 MB free)
  Uptime: 6.068533e6 sec
  Load Avg: 1.87060546875 1.5419921875 1.00341796875
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, skylake)

```

---

<div class="post-metadata">

**Author:** ![tautomer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tautomer/32/15268_2.png) [@tautomer](https://discourse.julialang.org/u/tautomer)\
**Post date:** [June 18, 2020, 9:35pm UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/38 "2020-06-18T21:35:22Z")

</div>

It seems MKL didn’t make too much difference in my case. I have a 3800x with all core at 4.25 GHz. The test is done in WSL2.

Version 1.4.1 installed from `apt`

```julia
julia> BLAS.openblas_get_config()
"OpenBLAS 0.3.7 USE64BITINT DYNAMIC_ARCH NO_AFFINITY Zen MAX_THREADS=16"

julia> BLAS.vendor()
:openblas64

julia> BLAS.set_num_threads(Sys.CPU_THREADS >> 1)

julia> LinearAlgebra.peakflops(16000)
3.133443793347056e11

julia> BLAS.set_num_threads(Sys.CPU_THREADS)

julia> LinearAlgebra.peakflops(16000)
4.313120502037118e11

julia> versioninfo(verbose=true)
Julia Version 1.4.1
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 20.04 LTS
  uname: Linux 4.19.84-microsoft-standard #1 SMP Wed Nov 13 11:44:37 UTC 2019 x86_64 x86_64
  CPU: AMD Ryzen 7 3800X 8-Core Processor:
                 speed user nice sys idle irq
       #1-16 4250 MHz 315688 s 9076 s 16522 s 5332624 s 0 s

  Memory: 24.99040985107422 GB (12217.40625 MB free)
  Uptime: 3548.0 sec
  Load Avg: 3.60107421875 3.52783203125 1.93408203125
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, znver1)
Environment:
  JULIA_NUM_THREADS = 4

```

Version 1.4.2, official binary.

```julia
julia> BLAS.openblas_get_config()
"OpenBLAS 0.3.5 USE64BITINT DYNAMIC_ARCH NO_AFFINITY Zen MAX_THREADS=32"

julia> BLAS.set_num_threads(Sys.CPU_THREADS >> 1)

julia> LinearAlgebra.peakflops(16000)
2.4798958833634494e11

julia> BLAS.set_num_threads(Sys.CPU_THREADS)

julia> LinearAlgebra.peakflops(16000)
3.476535665311297e11

```

The performance is a lot worse than `1.4.1`.

I used [MKL.jl](https://github.com/JuliaComputing/MKL.jl) to convert this binary to `MKL`.

Without setting `MKL_DEBUG_CPU_TYPE`

```julia
julia> BLAS.vendor()
:mkl

julia> BLAS.set_num_threads(Sys.CPU_THREADS >> 1)

julia> LinearAlgebra.peakflops(16000)
3.1590494395556305e11

julia> BLAS.set_num_threads(Sys.CPU_THREADS)

julia> LinearAlgebra.peakflops(16000)
3.1748345868355646e11

```

With `MKL_DEBUG_CPU_TYPE=5`

```julia
julia> BLAS.set_num_threads(Sys.CPU_THREADS >> 1)

julia> LinearAlgebra.peakflops(16000)
3.1601105540297565e11

BLAS.set_num_threads(Sys.CPU_THREADS)

julia> LinearAlgebra.peakflops(16000)
3.206082916145316e11

```

So `1.4.1` with `MKL` works best here.

It looks like `MKL` version won’t utilize all threads, even if `set_num_threads` is `16`.  
 ![Annotation 2020-06-18 173118](https://global.discourse-cdn.com/julialang/original/3X/5/1/519249584365e2d24c198daa1bfe308b3e6edb21.png)

With `OpenBLAS` and 16 threads.  
 ![Annotation 2020-06-18 173327](https://global.discourse-cdn.com/julialang/original/3X/9/f/9f22f845da8056692b4a0466a129455c9c965ae0.png)

I’m not sure if any one else has seen the same thing.

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [June 18, 2020, 9:56pm UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/39 "2020-06-18T21:56:59Z")

</div>

If you just want to benchmark OpenBLAS and MKL, I’d reccomend using `OpenBLAS_jll` and/or `MKL_jll`, as I do in [LoopVectorization’s benchmarks](https://github.com/chriselrod/LoopVectorization.jl/blob/93c4d7f87987e857a5e84f818c2467d288dd0dcb/benchmark/loadsharedlibs.jl#L64):

```julia
using MKL_jll, OpenBLAS_jll, Libdl, LinearAlgebra

const libMKL = Libdl.dlopen(MKL_jll.libmkl_rt)
const DGEMM_MKL = Libdl.dlsym(libMKL, :dgemm)
const SGEMM_MKL = Libdl.dlsym(libMKL, :sgemm)
const DGEMV_MKL = Libdl.dlsym(libMKL, :dgemv)
const MKL_SET_NUM_THREADS = Libdl.dlsym(libMKL, :MKL_Set_Num_Threads)

const libOpenBLAS = Libdl.dlopen(OpenBLAS_jll.libopenblas)
const DGEMM_OpenBLAS = Libdl.dlsym(libOpenBLAS, :dgemm_64_)
const SGEMM_OpenBLAS = Libdl.dlsym(libOpenBLAS, :sgemm_64_)
const DGEMV_OpenBLAS = Libdl.dlsym(libOpenBLAS, :dgemv_64_)
const OPENBLAS_SET_NUM_THREADS = Libdl.dlsym(libOpenBLAS, :openblas_set_num_threads64_)

istransposed(x) = 'N'
istransposed(x::Adjoint{<:Real}) = 'T'
istransposed(x::Adjoint) = 'C'
istransposed(x::Transpose) = 'T'
for (lib,f) ∈ [(:GEMM_MKL,:gemmmkl!), (:GEMM_OpenBLAS,:gemmopenblas!)]
    for (T,prefix) ∈ [(Float32,:S),(Float64,:D)]
        fm = Symbol(prefix, lib)
        @eval begin
            function $f(C::AbstractMatrix{$T}, A::AbstractMatrix{$T}, B::AbstractMatrix{$T})
                transA = istransposed(A)
                transB = istransposed(B)
                M, N = size(C); K = size(B, 1)
                pA = parent(A); pB = parent(B)
                ldA = stride(pA, 2)
                ldB = stride(pB, 2)
                ldC = stride(C, 2)
                α = one($T)
                β = zero($T)
                ccall(
                    $fm, Cvoid,
                    (Ref{UInt8}, Ref{UInt8}, Ref{Int64}, Ref{Int64}, Ref{Int64}, Ref{$T}, Ref{$T},
                     Ref{Int64}, Ref{$T}, Ref{Int64}, Ref{$T}, Ref{$T}, Ref{Int64}),
                    transA, transB, M, N, K, α, pA, ldA, pB, ldB, β, C, ldC
                )
            end
        end
    end
end
mkl_set_num_threads(N::Integer) = ccall(MKL_SET_NUM_THREADS, Cvoid, (Int32,), N % Int32)
openblas_set_num_threads(N::Integer) = ccall(OPENBLAS_SET_NUM_THREADS, Cvoid, (Int64,), N)

# if you want single threaded
mkl_set_num_threads(1) 
openblas_set_num_threads(1)

# if you want multithreaded
mkl_set_num_threads(Sys.CPU_THREADS)
openblas_set_num_threads(Sys.CPU_THREADS ÷ 2)
# Using the number of physical cores is fastest
# MKL automatically won't use more than this, therefore specifying `Sys.CPU_THREADS` should be safe
# OpenBLAS requires you to be exact. `Sys.CPU_THREADS ÷ 2` will yield that much of the time, but not all.

```

> The performance is a lot worse than `1.4.1` .

I’d guess that’s because `apt` uses OpenBLAS 0.3.7, while the official binary has OpenBLAS 0.3.5.  
The official binary for Julia 1.5beta ships OpenBLAS 0.3.9, and OpenBLAS\_jll should be able to provide this as well.

---

<div class="post-metadata">

**Author:** ![tautomer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tautomer/32/15268_2.png) [@tautomer](https://discourse.julialang.org/u/tautomer)\
**Post date:** [June 18, 2020, 11:11pm UTC](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033/40 "2020-06-18T23:11:05Z")

</div>

> [@Elrod](#):
>
> `using MKL_jll, OpenBLAS_jll`

Thanks a lot for the code, but `Libdl, LinearAlgebra` should also be added.

I tested your code with two 100x100 matrices. The results are like

1-thread MKL vs OpenBLAS: 45.600 μs vs 47.500 μs  
8-thread MKL vs OpenBLAS: 18.500 μs vs 24.800 μs  
16-thread MKL vs OpenBLAS: 17.600 μs vs 35.500 μs

Single thread is close. MKL scales better than OpenBLAS.

But for a fairly small matrix, say 10x10, OpenBLAS is faster. 447.475 vs 300 ns, 1 thread.

Hmm…for 1000x1000 matrices

1-thread MKL vs OpenBLAS: 34.320 ms vs 33.369 ms  
8-thread MKL vs OpenBLAS: 7.374 ms vs 9.142 ms

OpenBLAS is faster in single-thread again, but MKL is still ahead in multithread.

Anyway… I’m not doing intensive matrix operations at this moment, so I guess I can live with `LinearAlgebra` for now. Really appreciate your reply!

[Previous page](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033.md?page=1)

[Next page](https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-hardware-ryzen/8033.md?page=3)