# Regarding the multithreaded performance of OpenBLAS

**URL:** <https://discourse.julialang.org/t/regarding-the-multithreaded-performance-of-openblas/75450>\
**Category:** Performance\
**Tags:** blas, multithreading\
**Created:** [January 30, 2022, 4:48am UTC](https://discourse.julialang.org/t/regarding-the-multithreaded-performance-of-openblas/75450 "2022-01-30T04:48:52Z")\
**Posts on this page:** 1\
**Showing post:** 7

<div class="post-metadata">

**Author:** ![ash](https://avatars.discourse-cdn.com/v4/letter/a/f19dbf/32.png) [@ash](https://discourse.julialang.org/u/ash)\
**Post date:** [January 31, 2022, 5:10am UTC](https://discourse.julialang.org/t/regarding-the-multithreaded-performance-of-openblas/75450/7 "2022-01-31T05:10:22Z")

</div>

> [@carstenbauer](#):
>
> Depending on the computation / workload at hand “utilizing the whole machine” doesn’t necessarily mean 1600% CPU usage (if we take it that CPU usage is indicative of performance, which isn’t necessarily the case either). For example, if (the major part of) your computation is memory-bound (which is very often the case in scientific codes) you can add threads and enable more aggressive SIMD etc. as much as you want without changing much the utilization of you cores and the overall performance of your code. After all, that is what it means to be memory-bound rather than compute-bound 🙂 (See [here](https://github.com/carstenbauer/JuliaNRW21/blob/master/Day2/exercise_solutions/solution_optimization2.ipynb) for an hopefully instructive example.)

Thank you for explaining this to me so clearly. I hadn’t thought about that, I always thought memory bandwidth issues were more rare. What you said seems to describe my code perfectly.

While looking this up, I found [this](https://www.brendangregg.com/blog/2017-05-09/cpu-utilization-is-wrong.html)helpful post as well.

To test this for my code, I ran a little experiment. I set my RAM to different speeds, 4000CL36, 5200CL36, and 6000CL36. I benchmarked the read/write speeds and latencies for each configuration as well as two simulations. The results are below.

| | 4000CL36 64GB (Default) | 5200CL36 64GB (OC) | 6000CL36 32 GB (XMP) |
| --- | --- | --- | --- |
| mu (s) | 0.194 | 0.192 | 0.193 |
| beta (s) | 10.9 | 10.74 | 10.885 |
| Read (GB/s) | 62 | 81 | 92 |
| Write (GB/s) | 59 | 76 | 83 |
| Latency (ns) | 93 | 75 | 71 |

As you can see, the increase in RAM speed (which improves bandwidth, if I understand read/write speeds properly) does not improve the code’s performance. I am not sure what’s going on, I am probably benchmarking the wrong memory metric.

> [@carstenbauer](#):
>
> This has already been mentioned by me and @jpsamaroo in the [other discourse thread](https://discourse.julialang.org/t/matrix-multiplication-is-slower-when-multithreading-in-julia/56227/) you linked. Basically `OPENBLAS_NUM_THREADS=1` (you have a typo there) is special because it makes OpenBLAS computations run on the respective calling (Julia) thread. for `OPENBLAS_NUM_THREADS>1` the behavior changes qualitatively in the sense that OpenBLAS will create an own pool of OpenBLAS threads which it will use to run BLAS computation triggered by any of the Julia threads (there is only a single pool of OpenBLAS threads, irrespective of how many Julia threads you have). Hence, assuming `Threads.nthreads() == 16` , setting `OPENBLAS_NUM_THREADS=1` will, effectively, make all your BLAS computations run on all of the Julia threads (16) whereas setting `OPENBLAS_NUM_THREADS=2` will make all your BLAS computations run on only 2 separate OpenBLAS threads. That’s why you see such horrible performance for your 16/8 case for example.

I’d seen it mentioned in the other thread but honestly didn’t understand it completely, so thank you for this excellent explanation. Thank you for answering everything else as well.

---

_[View the full topic](https://discourse.julialang.org/t/regarding-the-multithreaded-performance-of-openblas/75450)._