Regarding the multithreaded performance of OpenBLAS

[ash]

@carstenbauer

Depending on the computation / workload at hand “utilizing the whole machine” doesn’t necessarily mean 1600% CPU usage (if we take it that CPU usage is indicative of performance, which isn’t necessarily the case either). For example, if (the major part of) your computation is memory-bound (which is very often the case in scientific codes) you can add threads and enable more aggressive SIMD etc. as much as you want without changing much the utilization of you cores and the overall performance of your code. After all, that is what it means to be memory-bound rather than compute-bound (See here for an hopefully instructive example.)

Thank you for explaining this to me so clearly. I hadn’t thought about that, I always thought memory bandwidth issues were more rare. What you said seems to describe my code perfectly.

While looking this up, I found thishelpful post as well.

To test this for my code, I ran a little experiment. I set my RAM to different speeds, 4000CL36, 5200CL36, and 6000CL36. I benchmarked the read/write speeds and latencies for each configuration as well as two simulations. The results are below.

	4000CL36 64GB (Default)	5200CL36 64GB (OC)	6000CL36 32 GB (XMP)
mu (s)	0.194	0.192	0.193
beta (s)	10.9	10.74	10.885
Read (GB/s)	62	81	92
Write (GB/s)	59	76	83
Latency (ns)	93	75	71

As you can see, the increase in RAM speed (which improves bandwidth, if I understand read/write speeds properly) does not improve the code’s performance. I am not sure what’s going on, I am probably benchmarking the wrong memory metric.

@carstenbauer

This has already been mentioned by me and @jpsamaroo in the other discourse thread you linked. Basically OPENBLAS_NUM_THREADS=1 (you have a typo there) is special because it makes OpenBLAS computations run on the respective calling (Julia) thread. for OPENBLAS_NUM_THREADS>1 the behavior changes qualitatively in the sense that OpenBLAS will create an own pool of OpenBLAS threads which it will use to run BLAS computation triggered by any of the Julia threads (there is only a single pool of OpenBLAS threads, irrespective of how many Julia threads you have). Hence, assuming Threads.nthreads() == 16 , setting OPENBLAS_NUM_THREADS=1 will, effectively, make all your BLAS computations run on all of the Julia threads (16) whereas setting OPENBLAS_NUM_THREADS=2 will make all your BLAS computations run on only 2 separate OpenBLAS threads. That’s why you see such horrible performance for your 16/8 case for example.

I’d seen it mentioned in the other thread but honestly didn’t understand it completely, so thank you for this excellent explanation. Thank you for answering everything else as well.