# Multithreading using more CPUs than expected

**URL:** <https://discourse.julialang.org/t/multithreading-using-more-cpus-than-expected/101599>\
**Category:** Performance\
**Created:** [July 14, 2023, 3:27am UTC](https://discourse.julialang.org/t/multithreading-using-more-cpus-than-expected/101599 "2023-07-14T03:27:34Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![lzxnl](https://avatars.discourse-cdn.com/v4/letter/l/ce7236/32.png) [@lzxnl](https://discourse.julialang.org/u/lzxnl)\
**Post date:** [July 14, 2023, 3:27am UTC](https://discourse.julialang.org/t/multithreading-using-more-cpus-than-expected/101599/1 "2023-07-14T03:27:34Z")

</div>

I opened a Jupyter Notebook kernel with 25 threads (`threadid()` prints out 25 in the beginning). I then proceed to run some code that manages to use 8900% CPU, as recorded by `top` on the terminal, i.e. 89 threads worth of CPU, even though I’m only asking for 25 threads. Do people have any suggestions? For reference, I just want to run a sequence of for loops over a 5 dimensional array, and I’ve been doing something like

```julia
ThreadPools.@qthreads for i in 1:6
ThreadPools.@qthreads for j in 1:4
ThreadPools.@qthreads for k in 1:5

```

etc. How is the system using 8900% CPU despite only having 25 threads called in the beginning?

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [July 14, 2023, 3:52am UTC](https://discourse.julialang.org/t/multithreading-using-more-cpus-than-expected/101599/2 "2023-07-14T03:52:21Z")

</div>

BLAS threads (i.e. for matrix multiplication) are separate from Julia threads.

---

<div class="post-metadata">

**Author:** ![lzxnl](https://avatars.discourse-cdn.com/v4/letter/l/ce7236/32.png) [@lzxnl](https://discourse.julialang.org/u/lzxnl)\
**Post date:** [July 14, 2023, 4:02am UTC](https://discourse.julialang.org/t/multithreading-using-more-cpus-than-expected/101599/3 "2023-07-14T04:02:22Z")

</div>

I heard you can stop BLAS from using more threads by calling julia -p 1, for instance. If I run my code without the parallelised loops, and on julia -p 1 (supposedly shutting down the BLAS extra threads), with one process and one thread, it still uses 6400% CPU. What could be going on?

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [July 14, 2023, 4:40am UTC](https://discourse.julialang.org/t/multithreading-using-more-cpus-than-expected/101599/4 "2023-07-14T04:40:37Z")

</div>

You need to use `LinearAlgebra.BLAS.set_num_threads(1)` to set the number of BLAS threads

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [July 14, 2023, 5:01am UTC](https://discourse.julialang.org/t/multithreading-using-more-cpus-than-expected/101599/5 "2023-07-14T05:01:04Z")

</div>

> [@lzxnl](#):
>
> I heard you can stop BLAS from using more threads by calling julia -p 1, for instance.

That’s wrong. Use the `OPENBLAS_NUM_THREADS=1` environment variable or the interactive option that @jishnub suggested.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [July 14, 2023, 6:37am UTC](https://discourse.julialang.org/t/multithreading-using-more-cpus-than-expected/101599/6 "2023-07-14T06:37:45Z")

</div>

For more details, check out this recent addition to the docs: [Performance tips - multithreading and linear algebra](https://docs.julialang.org/en/v1.11-dev/manual/performance-tips/#man-multithreading-linear-algebra)

---

<div class="post-metadata">

**Author:** ![lzxnl](https://avatars.discourse-cdn.com/v4/letter/l/ce7236/32.png) [@lzxnl](https://discourse.julialang.org/u/lzxnl)\
**Post date:** [July 14, 2023, 12:41pm UTC](https://discourse.julialang.org/t/multithreading-using-more-cpus-than-expected/101599/7 "2023-07-14T12:41:17Z")

</div>

> [@jishnub](#):
>
> LinearAlgebra.BLAS.set\_num\_threads(1)

Thanks! This worked! Wow I never knew BLAS would be calling up to 64 threads on its own.

---

<div class="post-metadata">

**Author:** ![lzxnl](https://avatars.discourse-cdn.com/v4/letter/l/ce7236/32.png) [@lzxnl](https://discourse.julialang.org/u/lzxnl)\
**Post date:** [July 15, 2023, 4:09am UTC](https://discourse.julialang.org/t/multithreading-using-more-cpus-than-expected/101599/8 "2023-07-15T04:09:37Z")

</div>

Somehow, the problem came back, even without BLAS, and regardless of if I run the program in the terminal, or Jupyter Notebook, the CPU usage keeps blowing up, even if I only ask for 25 cores. Does anyone know what could be causing this? I am only using matrix calculations in my code, with `LinearAlgebra.BLAS.set_num_threads(1)` set.

The kernel appears to die immediately after the @threads call, which is really strange. If I run it on Jupyter Notebook, it says ‘the kernel appears to have died’. If I run it on the terminal, it says ‘Killed’.

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [July 15, 2023, 6:07am UTC](https://discourse.julialang.org/t/multithreading-using-more-cpus-than-expected/101599/9 "2023-07-15T06:07:19Z")

</div>

Could you try using the environment variable? Does that also lead to excessive usage? Could you also print out `BLAS.get_num_threads()` before the threaded loop?

---

<div class="post-metadata">

**Author:** ![lzxnl](https://avatars.discourse-cdn.com/v4/letter/l/ce7236/32.png) [@lzxnl](https://discourse.julialang.org/u/lzxnl)\
**Post date:** [July 20, 2023, 3:23am UTC](https://discourse.julialang.org/t/multithreading-using-more-cpus-than-expected/101599/10 "2023-07-20T03:23:14Z")

</div>

I eventually found the problem. Turns out that I needed to call garbage collection every loop iteration, after setting the intermediate buffer vectors to `nothing`.

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [July 20, 2023, 4:10am UTC](https://discourse.julialang.org/t/multithreading-using-more-cpus-than-expected/101599/11 "2023-07-20T04:10:24Z")

</div>

This shouldn’t be necessary in general. Could you post a minimal example that leads to this? It sounds like a julia issue

---

<div class="post-metadata">

**Author:** ![danielwe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/danielwe/32/35657_2.png) [@danielwe](https://discourse.julialang.org/u/danielwe)\
**Post date:** [July 20, 2023, 5:10am UTC](https://discourse.julialang.org/t/multithreading-using-more-cpus-than-expected/101599/12 "2023-07-20T05:10:57Z")

</div>

> [@lzxnl](#):
>
> Turns out that I needed to call garbage collection every loop iteration

If this makes a major difference you’re likely allocating a lot within your multithreaded tasks. Be aware that multithreading scales poorly in such cases due to single-threaded GC (at least until Julia 1.10 drops). To avoid this, you should try to allocate intermediate buffers once per task rather than once per iteration. This blog post shows one way to do it (dropping `@threads` in favor of upfront chunking and `@spawn`): [PSA: Thread-local state is no longer recommended](https://julialang.org/blog/2023/07/PSA-dont-use-threadid/#better_fix_work_directly_with_tasks).

* * *

On a different note, be aware that `ThreadPools` seems to not quite have kept up with the changes required by the dynamic schedule that `@threads` defaults to since Julia 1.8. But this new schedule also reduces the need for `ThreadPools`, so I’d suggest working without it, using just `@threads` and/or `@spawn`, and only bringing back `ThreadPools` once you’re sure your code works, if you’re still curious about its functionality.

(Specifically, while `ThreadPools.@tspawnat` was fixed, I’ve noticed other places in the code where `@threads` is used assuming the semantics of `@threads :static`. I’ve been meaning to file an issue, but haven’t gotten around to it yet.)