# BLAS thread count vs Julia thread count

**URL:** <https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197>\
**Category:** General Usage\
**Tags:** question, performance, linearalgebra\
**Created:** [March 15, 2021, 3:44am UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197 "2021-03-15T03:44:19Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Manuel\_Bergmann](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/manuel_bergmann/32/20174_2.png) [@Manuel\_Bergmann](https://discourse.julialang.org/u/Manuel_Bergmann)\
**Post date:** [March 15, 2021, 3:44am UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/1 "2021-03-15T03:44:19Z")

</div>

Hey, just found that thread now. Is there a reason why `BLAS.set_num_threads(n)` defaults to 8 threads and also ignores Julia’s `--threads` option? I found that issue while investigating that matrix multiplication benchmark: [https://github.com/kostya/benchmarks/issues/312](https://github.com/kostya/benchmarks/issues/312) - the benchmark does not set the number of threads and BLAS defaults to 8 on the 12 thread test machine while numpy uses all threads. Same on my machine: It defaults to 8 threads as well, while my CPU has 16 threads and load does not exceed 60%. I tried to set it with the `--threads` option and then with the regular thread environment variable first. It would probably be a less tricky thing to acknowledge those options when set and use `BLAS.set_num_threads(n)` only to override those settings(?)

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [March 15, 2021, 10:49am UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/2 "2021-03-15T10:49:39Z")

</div>

> [@Manuel\_Bergmann](#):
>
> Is there a reason why `BLAS.set_num_threads(n)` defaults to 8 threads and also ignores Julia’s `--threads` option?

BLAS’ threading is completely independent from Julia’s threading, there is no reason to tie them together.

---

<div class="post-metadata">

**Author:** ![paulmelis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulmelis/32/35063_2.png) [@paulmelis](https://discourse.julialang.org/u/paulmelis)\
**Post date:** [March 15, 2021, 11:55am UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/3 "2021-03-15T11:55:25Z")

</div>

I also ran into the BLAS auto-threading recently and was quite surprised that it isn’t limited by the `-t` option. One way I viewed Julia’s `-t` option so far is that it limits the number of threads used by a Julia program. This can be useful in situations where you want to run multiple Julia processes on a multi-core system, without interference due to too many threads being used in total.

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [March 15, 2021, 12:00pm UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/4 "2021-03-15T12:00:17Z")

</div>

One issue with tying them together is that you might want to limit the number of Julia threads in order to get the most out of threading with the BLAS library, or vice-versa (limit BLAS threads since you plan to use them more effectively on the Julia side). So often if one is small you might want the other to be large. So I don’t think it makes sense to tie them to the same flag.

So maybe it’s more of a documentation issue than a “what should `-t` do” issue. E.g. [Multi-Threading · The Julia Language](https://docs.julialang.org/en/v1/manual/multi-threading/) doesn’t mention BLAS at all.

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [March 15, 2021, 12:21pm UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/5 "2021-03-15T12:21:03Z")

</div>

Clearly, the solution to this is having our own super fast Julia-BLAS (see Octavian.jl etc.). 🙂

---

<div class="post-metadata">

**Author:** ![Manuel\_Bergmann](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/manuel_bergmann/32/20174_2.png) [@Manuel\_Bergmann](https://discourse.julialang.org/u/Manuel_Bergmann)\
**Post date:** [March 15, 2021, 1:03pm UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/6 "2021-03-15T13:03:52Z")

</div>

> [@ericphanson](#):
>
> One issue with tying them together is that you might want to limit the number of Julia threads in order to get the most out of threading with the BLAS library, or vice-versa (limit BLAS threads since you plan to use them more effectively on the Julia side). So often if one is small you might want the other to be large. So I don’t think it makes sense to tie them to the same flag.
> 
> So maybe it’s more of a documentation issue than a “what should `-t` do” issue. E.g. [Multi-Threading · The Julia Language](https://docs.julialang.org/en/v1/manual/multi-threading/) doesn’t mention BLAS at all.

I think, like I wrote before, completely tying them together wouldn’t be necessary: One could use the `BLAS` setting as an override to the `--threads` option. I.e.:

- `--threads` not set and `BLAS.set_num_threads(n)` not set: Use either all available threads for BLAS or maybe just 1 thread/Julias default number of threads to keep it consistent. Do not use 8 threads regardless of the machine architecture by default.
- `--threads` set and `BLAS.set_num_threads(n)` not set: Use the number of threads passed to `--threads` for everything including BLAS
- `--threads` not set and `BLAS.set_num_threads(n)` set: Use the number of threads set for BLAS for BLAS and for the rest of Julia the defaults (=behaviour like it is now)
- `--threads` set and `BLAS.set_num_threads(n)` set: Use `--threads` for Julia and the BLAS option for BLAS (=behavior like it is now)

This way you have one or two pitfalls less and are still able to control both thread counts individually if you feel like doing so.

I just think the current behaviour of Julia is counterintuitive and BLAS defaulting to use 8 threads regardless of the machine architecture also seems like an odd choice and a pitfall. I read somewhere here that this was done because most users had 4 core machines with 8 threads so 8 threads was chosen, but why not simply fetch the thread count of the current machine and use that instead like numpy seems to be doing?

---

<div class="post-metadata">

**Author:** ![Manuel\_Bergmann](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/manuel_bergmann/32/20174_2.png) [@Manuel\_Bergmann](https://discourse.julialang.org/u/Manuel_Bergmann)\
**Post date:** [March 15, 2021, 1:17pm UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/7 "2021-03-15T13:17:41Z")

</div>

> [@carstenbauer](#):
>
> Clearly, the solution to this is having our own super fast Julia-BLAS (see Octavian.jl etc.). 🙂

If those are all faster than Julias internal implementation and circumvent those issues is there a reason to not make any of those the default implementation in Julia? (As you might have guessed it, I am new to Julia and Julia’s community)

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [March 15, 2021, 1:22pm UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/8 "2021-03-15T13:22:30Z")

</div>

Architecture specific optimizations and larger matrices are (as far as I know) still places where regular BLAS shines. It’d also be a considerable amount of effort to integrate one of those packages into Base and make sure it’s all compatible and doesn’t introduce regressions.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [March 15, 2021, 1:30pm UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/9 "2021-03-15T13:30:23Z")

</div>

That said, I think it’s reasonably likely that 2 years from now we will have this integrated. The biggest challenge is that doing this requires all of the stack for it be very stable.

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [March 15, 2021, 2:00pm UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/10 "2021-03-15T14:00:20Z")

</div>

> [@Manuel\_Bergmann](#):
>
> I just think the current behaviour of Julia is counterintuitive and BLAS defaulting to use 8 threads regardless of the machine architecture also seems like an odd choice and a pitfall.

I don’t think that’s the case? According to [BLAS threads should default to physical not logical core count? · Issue #33409 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/33409#issuecomment-536144941) it chooses it based on `Sys.CPU_THREADS` (and that issue is about how it should be based on the number of cores, not the number of threads).

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [March 15, 2021, 2:24pm UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/11 "2021-03-15T14:24:10Z")

</div>

It does max out at 8 though.

```julia
julia> BLAS.get_num_threads()
8

julia> Hwloc.num_physical_cores()
20

```

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [March 15, 2021, 2:37pm UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/12 "2021-03-15T14:37:21Z")

</div>

Also, welcome @Manuel_Bergmann! I think its worth splitting this topic into a dedicated discussion since it’s valuable in its own right.

---

<div class="post-metadata">

**Author:** ![cgeoga](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cgeoga/32/216186_2.png) [@cgeoga](https://discourse.julialang.org/u/cgeoga)\
**Post date:** [March 15, 2021, 4:25pm UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/13 "2021-03-15T16:25:00Z")

</div>

I’m not the most qualified person to comment on this, but perhaps part of the reason that they are separate is because BLAS’s scheduler (?) and Julia’s scheduler (?) don’t currently talk to each other in the way that two threaded pieces of Julia code somehow compose efficiently magically and for free (from the perspective of an end-user like me). Another word that I don’t understand but that seems to come up is partr.

I’ve been bitten several times by, for example, wanting to thread-map a bunch of `cholesky!` calls to a `Vector{Symmetric{...}}`. It was meaningfully slower than the single threaded map call, and after some help from @tro3 and the excellent logging abilities of `ThreadPools.jl` I came to understand that the BLAS threading stuff and the Julia threading stuff don’t really play nicely with each other. So at the very least until that’s resolved, I absolutely love being able to `BLAS.set_num_threads(1)` when I want to `ThreadPools.tmap(cholesky!, large_vec_of_small_matrices)`, and then crank the `BLAS` threads back up when I’m working with big matrices. But it’s definitely a gotcha until you realize what’s going on.

---

<div class="post-metadata">

**Author:** ![paulmelis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulmelis/32/35063_2.png) [@paulmelis](https://discourse.julialang.org/u/paulmelis)\
**Post date:** [March 15, 2021, 4:29pm UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/14 "2021-03-15T16:29:52Z")

</div>

> [@ericphanson](#):
>
> So maybe it’s more of a documentation issue than a “what should `-t` do” issue. E.g. [Multi-Threading · The Julia Language](https://docs.julialang.org/en/v1/manual/multi-threading/) doesn’t mention BLAS at all.

I like the principle of least surprise and BLAS indeed violates it a bit. Also, the BLAS multi-threading is fundamental to many packages using operations on (large) matrices, yet those are not all uses of Julia I would expect. Can’t hurt to make a mention of it in the docs.

---

<div class="post-metadata">

**Author:** ![jlperla](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jlperla/32/34332_2.png) [@jlperla](https://discourse.julialang.org/u/jlperla)\
**Post date:** [March 15, 2021, 4:52pm UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/15 "2021-03-15T16:52:21Z")

</div>

Related to the thread counts and configuation of blas. I have noticed that there is a note on BLAS in [https://github.com/JuliaLang/julia/blob/master/NEWS.md#linearalgebra](https://github.com/JuliaLang/julia/blob/master/NEWS.md#linearalgebra) and in particular [https://github.com/staticfloat/libblastrampoline/](https://github.com/staticfloat/libblastrampoline/)

Does this mean that in 1.7 it will be easy to swap out BLAS implementations instead of relying on MKL.jl/etc.?

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [March 15, 2021, 4:53pm UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/16 "2021-03-15T16:53:13Z")

</div>

That’s the idea. With a little luck `Octavian` (BLAS in Julia) will also be easy to plug in.

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [March 15, 2021, 5:07pm UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/17 "2021-03-15T17:07:29Z")

</div>

> [@jlperla](#):
>
> instead of relying on MKL.jl/etc.

Well, `using MKL` will be the way to dynamically switch to MKL. But yes, it’s entirely different to what MKL.jl did before.

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [March 15, 2021, 6:37pm UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/18 "2021-03-15T18:37:17Z")

</div>

Another option to note here: Strided.jl has an option to replace the BLAS threads with Julia’s threads: [GitHub - Jutho/Strided.jl: A Julia package for strided array views and efficient manipulations thereof](https://github.com/Jutho/Strided.jl#whats-new) .

---

<div class="post-metadata">

**Author:** ![Manuel\_Bergmann](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/manuel_bergmann/32/20174_2.png) [@Manuel\_Bergmann](https://discourse.julialang.org/u/Manuel_Bergmann)\
**Post date:** [March 15, 2021, 8:04pm UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/19 "2021-03-15T20:04:40Z")

</div>

> [@ericphanson](#):
>
> I don’t think that’s the case? According to [BLAS threads should default to physical not logical core count? · Issue #33409 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/33409#issuecomment-536144941) it chooses it based on `Sys.CPU_THREADS` (and that issue is about how it should be based on the number of cores, not the number of threads).

> [@mbauman](#):
>
> It does max out at 8 though.
> 
> ```julia
> julia> BLAS.get_num_threads()
> 8
> 
> julia> Hwloc.num_physical_cores()
> 20
> 
> ```

I believe this conversation underlines my point pretty well. And that behavior (if I got this right: it automatically chooses the correct number of threads but only up to 8 and then it just chooses 8) honestly sounds even worse than expected, because it won’t be an issue until you try your code on a machine with more than 8 cores and it can be really tricky to identify. It is yet another thing you simply need to know and it’s really not intuitive. I assume, most people will just run straight into that, wasting time searching for why their code does not fully utilize their CPU. Especially beginners who want to play with the language and find out if they want to stick with it. But from the conversation above it seems that this is not something even more experienced people necessarily seem to be aware of (the upper limit).  
Side note: Maybe explaining this issue should go into the [performance tips](https://docs.julialang.org/en/v1/manual/performance-tips/) ? I think it would be a good fit there.

I think this discussion shouldn’t be, how easy it is to circumvent the problem, but how many beginners, students etc. (like me) will run into this issue. If the mechanism of how different BLAS packages are loaded will change in Julia 1.7, this might be a good opportunity to change this point as well(?)

---

<div class="post-metadata">

**Author:** ![juthohaegeman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juthohaegeman/32/8620_2.png) [@juthohaegeman](https://discourse.julialang.org/u/juthohaegeman)\
**Post date:** [March 15, 2021, 8:27pm UTC](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197/20 "2021-03-15T20:27:57Z")

</div>

> [@Manuel\_Bergmann](#):
>
> I believe this conversation underlines my point pretty well. And that behavior (if I got this right: it automatically chooses the correct number of threads but only up to 8 and then it just chooses 8)

I think the reason that it maxes out at 8 is due to OpenBLAS, not Julia. It probably does not happen if you use MKL instead. I don’t know the reason for this, maybe the OpenBLAS threading becomes inefficient above 8 threads.

[Next page](https://discourse.julialang.org/t/blas-thread-count-vs-julia-thread-count/57197.md?page=2)