# Is the best number of threads used in parallel computing by using distribute 4?

**URL:** <https://discourse.julialang.org/t/is-the-best-number-of-threads-used-in-parallel-computing-by-using-distribute-4/40139>\
**Category:** Performance\
**Tags:** parallel\
**Created:** [May 25, 2020, 4:01pm UTC](https://discourse.julialang.org/t/is-the-best-number-of-threads-used-in-parallel-computing-by-using-distribute-4/40139 "2020-05-25T16:01:10Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![MTone](https://avatars.discourse-cdn.com/v4/letter/m/f07891/32.png) [@MTone](https://discourse.julialang.org/u/MTone)\
**Post date:** [May 25, 2020, 4:01pm UTC](https://discourse.julialang.org/t/is-the-best-number-of-threads-used-in-parallel-computing-by-using-distribute-4/40139/1 "2020-05-25T16:01:10Z")

</div>

I am doing a numerical calculation using Julia, and recently I just tested the speed of my code using parallel computing by applying @distributed in the summation:  
 ![Julia1](https://global.discourse-cdn.com/julialang/original/3X/4/4/44994b89a9fade0be7d1dbff3b4165e6327511c7.png)  
(It is just part of my code. The attached part is enclosed in another loop so Sm is a local variable and can be used in \eta loop directly. I am sure my code is correct and gives correct results.)  
Then I collected the running time for using different number of threads (1, 4, 8, 16 and 32):  
(9:52, 7:14, 6:54, 6:50 and 7:34 in terms of “HH:MM”, respectively)  
It looks for me that when I just from 1 thread to 4 threads the improvement is best. With the threads number increasing, the performance doesn’t increase that much and for using 32 threads, it even took longer time than using 4 threads.  
I am sure that I allocated enough number of threads when I running this code on my cluster. I used “export JULIA\_NUM\_THREADS=Value” (Value = 1, 4, 8, 16, 32) in my scripts to ensure that Julia will start with this many threads. I also used:  
"N\_t = nthreads()  
println("Number of Threads = “,N\_t)”  
in my code to check and confirm that I did allocate this much threads.  
So my question is:

1. Does anyone else test or feel that the parallel computing by @distributed in Julia is not linear related to the number of threads? Is my case special or common?
2. If my case is a common case, is there any possible explanation on why the running time is not 1/4 with 4 times of threads and why when using 4 threads it seems to get the best improvement?
3. I am very new to Julia and even for programming. May I know if another languages have similar performance in parallel computing?

Any answer is welcome. Thanks for your reading and replying

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [May 25, 2020, 4:11pm UTC](https://discourse.julialang.org/t/is-the-best-number-of-threads-used-in-parallel-computing-by-using-distribute-4/40139/2 "2020-05-25T16:11:07Z")

</div>

Linear scaling is the best possible outcome (not really, it _is_ possible to get superlinear scaling). It is rarely observed. The speedup you will see depends on your architecture and the nature of your program. There are not really any universal, clear-cut truths to parallel programming that I know of.

---

<div class="post-metadata">

**Author:** ![MTone](https://avatars.discourse-cdn.com/v4/letter/m/f07891/32.png) [@MTone](https://discourse.julialang.org/u/MTone)\
**Post date:** [May 25, 2020, 4:34pm UTC](https://discourse.julialang.org/t/is-the-best-number-of-threads-used-in-parallel-computing-by-using-distribute-4/40139/3 "2020-05-25T16:34:10Z")

</div>

Thank you tbeason! I used to think that linear scaling is universal.

---

<div class="post-metadata">

**Author:** ![lungben](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lungben/32/12314_2.png) [@lungben](https://discourse.julialang.org/u/lungben)\
**Post date:** [May 25, 2020, 7:35pm UTC](https://discourse.julialang.org/t/is-the-best-number-of-threads-used-in-parallel-computing-by-using-distribute-4/40139/4 "2020-05-25T19:35:59Z")

</div>

`@distributed` is for multiprocessing, whereas the `JULIA_NUM_THREADS` environment variable is for multithreading.  
For multiprocessing with `@distributed`, use `addprocs()` to add worker processes.  
Alternatively, use `Threads.@threads` or `Threads.@spawn` for multithreading.

---

<div class="post-metadata">

**Author:** ![MTone](https://avatars.discourse-cdn.com/v4/letter/m/f07891/32.png) [@MTone](https://discourse.julialang.org/u/MTone)\
**Post date:** [June 11, 2020, 4:56pm UTC](https://discourse.julialang.org/t/is-the-best-number-of-threads-used-in-parallel-computing-by-using-distribute-4/40139/5 "2020-06-11T16:56:33Z")

</div>

Thank you lungben! I rewrite my code with `addprocs()` to add processors and my code become much faster. I just noticed that previously, I ran my code on only one processor.  
Also, I realized that there are 2 ways of parallel computing in Julia. One is `Threads.@threads` or `Threads.@spawn` used with `$env:JULIA_NUM_THREADS = <nthreads>` which with shared memory. Another one is `Distributed@distributed` with `addprocs()` for multiprocessing which will distribute the job into several processors who has their only memories.