# Parallelization on the CPU isn't effective

**URL:** <https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795>\
**Category:** General Usage\
**Created:** [November 19, 2021, 7:48pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795 "2021-11-19T19:48:25Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![leopdsf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/leopdsf/32/31008_2.png) [@leopdsf](https://discourse.julialang.org/u/leopdsf)\
**Post date:** [November 19, 2021, 7:48pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/1 "2021-11-19T19:48:25Z")

</div>

Hello, everyone. I’m following the tutorial introduction to GPU programming in Julia ([Introduction · CUDA.jl](https://cuda.juliagpu.org/stable/tutorials/introduction/#Your-first-GPU-computation)), and the first example in the tutorial discuss parallelization on CPU. Didn’t find a specific section for CPU, so thought General Usage would be addequate. The code is

```julia
using Test
using BenchmarkTools

N=2^20
x = fill(1.0f0, N) # a vector filled with 1.0 (Float32)
y = fill(2.0f0, N)

function sequential_add!(y, x)
    for i in eachindex(y, x)
        @inbounds y[i] += x[i]
    end
    return nothing
end

function parallel_add!(y, x)
    Threads.@threads for i in eachindex(y, x)
        @inbounds y[i] += x[i]
    end
    return nothing
end

@btime sequential_add!($y, $x)
@btime parallel_add!($y, $x)

```

The results given by the tutorial for the benchmarking with 4 threads of function `sequential_add` is `487.303 μs (0 allocations: 0 bytes)`, and for the function `parallel_add` is `259.587 μs (13 allocations: 1.48 KiB)`, showing that parallel computation is faster.

When I run the code on my PC, the execution time is roughly the same (`387.601 μs (0 allocations: 0 bytes)` for parallel and `386.501 μs (20 allocations: 3.39 KiB)`), and running `Threads.nthreads()`, the result is 4, therefore I am also using 4 threads.

This should be a problem, right? Am I doing something wrong while compiling, or wrongly interpreting the use of threads? Also, there is some package that I need to install before parallelizing?

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [November 19, 2021, 7:54pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/2 "2021-11-19T19:54:56Z")

</div>

That’s strange. Here is what I get:

```julia
julia> @btime sequential_add!($y, $x)
  250.849 μs (0 allocations: 0 bytes)

julia> @btime parallel_add!($y, $x)
  81.248 μs (21 allocations: 1.83 KiB)

```

To answer your question, no you don’t need to install a package or do anything special expect for making sure to start Julia with 4 threads, i.e. `julia -t4`.

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [November 19, 2021, 7:58pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/3 "2021-11-19T19:58:44Z")

</div>

OP reports `nthreads()` being 4. So another explanation would be that some of these threads are not available;)

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [November 19, 2021, 8:08pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/4 "2021-11-19T20:08:28Z")

</div>

Yeah, I guess in principle it could be a coincidence (e.g. caused by other OS threads keeping the cores busy). However, the timings are so similar that it’s way more likely that the benchmarks have been accidentally run with `nthreads() == 1`.

Perhaps the number of threads was 4 at some point but the OP has restarted Julia while experimenting around and forgotten to start Julia with 4 threads again (just guessing of course). @leopdsf could you ensure that this was not the case? Perhaps you could post a screenshot of your terminal like this, i.e. including the command to start Julia:

 ![Screenshot 2021-11-19 at 21.08.07](https://global.discourse-cdn.com/julialang/original/3X/f/c/fcd1164be166e89db0575181feea69ecd2b65cc6.png)

---

<div class="post-metadata">

**Author:** ![leopdsf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/leopdsf/32/31008_2.png) [@leopdsf](https://discourse.julialang.org/u/leopdsf)\
**Post date:** [November 19, 2021, 8:16pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/5 "2021-11-19T20:16:28Z")

</div>

Hello @carstenbauer and @goerch, thanks in advance for answering. Here are the prints:

 ![P1](https://global.discourse-cdn.com/julialang/original/3X/d/c/dc84ef960ac29f7ea7ce2f1e74e621377cc68594.png)  
 ![P2](https://global.discourse-cdn.com/julialang/original/3X/c/f/cf66175f13cd8ac7d71a2af8bbda40b65456a5f8.png)

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [November 19, 2021, 8:20pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/6 "2021-11-19T20:20:43Z")

</div>

Hm, can you post the output of `Sys.CPU_THREADS` please?

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [November 19, 2021, 8:21pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/7 "2021-11-19T20:21:15Z")

</div>

I don’t see a reference to `Threads.nthreads()`?

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [November 19, 2021, 8:22pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/8 "2021-11-19T20:22:39Z")

</div>

Yeah, I also missed it my screenshot above. Would be good to explicitly confirm it, but he is clearly starting Julia with `-t4` (and the command line argument has higher precedence than any potententially set env variable).

---

<div class="post-metadata">

**Author:** ![leopdsf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/leopdsf/32/31008_2.png) [@leopdsf](https://discourse.julialang.org/u/leopdsf)\
**Post date:** [November 19, 2021, 8:23pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/9 "2021-11-19T20:23:43Z")

</div>

Here they are:  
 ![P3](https://global.discourse-cdn.com/julialang/original/3X/d/c/dcfa7eb3818022eae735e1fc958d8c7191970667.png)

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [November 19, 2021, 8:24pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/10 "2021-11-19T20:24:34Z")

</div>

Wow, that looks really bad. Is your system doing something else? What does your system monitor show? What does `versioninfo()` show?

---

<div class="post-metadata">

**Author:** ![leopdsf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/leopdsf/32/31008_2.png) [@leopdsf](https://discourse.julialang.org/u/leopdsf)\
**Post date:** [November 19, 2021, 8:30pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/11 "2021-11-19T20:30:47Z")

</div>

The first is in portuguese, trying to put it in english

 ![P4](https://global.discourse-cdn.com/julialang/original/3X/4/5/45215a1bb41eaad193b3be1fd8d90924967dfa0a.png)  
 ![p5](https://global.discourse-cdn.com/julialang/original/3X/9/d/9d07f0e5bdf796cb8715d3c94ffdc4f9015f7477.png)

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [November 19, 2021, 8:35pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/12 "2021-11-19T20:35:14Z")

</div>

Thanks for your help. Here is my info running the example from an `include`d file

```julia
  144.400 μs (0 allocations: 0 bytes)
  37.000 μs (30 allocations: 3.62 KiB)

```

and `versioninfo()` giving

```julia
Julia Version 1.7.0-rc3
Commit 3348de4ea6 (2021-11-15 08:22 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-10710U CPU @ 1.10GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, skylake)
Environment:
  JULIA_NUM_THREADS = 6

```

So maybe the simplest option would be to rerun the test on a newer version of Julia?

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [November 19, 2021, 8:37pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/13 "2021-11-19T20:37:03Z")

</div>

It’s not the Julia version. (I would know but to be sure I’ve explicitly run the example in 1.5, 1.6 and 1.7, all with the expected speedup.) But of course, it can’t hurt to update at least to 1.6.

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [November 19, 2021, 8:38pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/14 "2021-11-19T20:38:23Z")

</div>

So is it a package then?

---

<div class="post-metadata">

**Author:** ![gbaraldi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gbaraldi/32/22101_2.png) [@gbaraldi](https://discourse.julialang.org/u/gbaraldi)\
**Post date:** [November 19, 2021, 8:46pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/15 "2021-11-19T20:46:35Z")

</div>

if you got to task manager and check the by cpu graph do all cores get loaded or just one?

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [November 19, 2021, 8:47pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/16 "2021-11-19T20:47:15Z")

</div>

OP just showed it to us above.

---

<div class="post-metadata">

**Author:** ![gbaraldi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gbaraldi/32/22101_2.png) [@gbaraldi](https://discourse.julialang.org/u/gbaraldi)\
**Post date:** [November 19, 2021, 8:49pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/17 "2021-11-19T20:49:33Z")

</div>

But that’s a generalized graph, not sure if 100% there means a single core loaded to the max or all cores loaded to the max. I get 100% and like 380% on my computer indicating 4 cores working .

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [November 19, 2021, 8:50pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/18 "2021-11-19T20:50:57Z")

</div>

You could try to setup this example in a separate environment [4. Working with Environments · Pkg.jl](https://pkgdocs.julialang.org/v1.2/environments/) to exclude influence of installed packages.

---

<div class="post-metadata">

**Author:** ![leopdsf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/leopdsf/32/31008_2.png) [@leopdsf](https://discourse.julialang.org/u/leopdsf)\
**Post date:** [November 19, 2021, 10:21pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/19 "2021-11-19T22:21:54Z")

</div>

The 4 cores are working, as shown in the print. Updating to 1.6.3 also didn’t help. I will try using the environments now. For now, thank you very much guys!

 ![image](https://global.discourse-cdn.com/julialang/original/3X/c/2/c23e8d8b248d52dfe0210f12f31e0b03fe071113.png)

---

<div class="post-metadata">

**Author:** ![Jeff\_Emanuel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeff_emanuel/32/15440_2.png) [@Jeff\_Emanuel](https://discourse.julialang.org/u/Jeff_Emanuel)\
**Post date:** [November 19, 2021, 10:29pm UTC](https://discourse.julialang.org/t/parallelization-on-the-cpu-isnt-effective/71795/20 "2021-11-19T22:29:52Z")

</div>

Grasping at straws here, but check Windows power settings. Maybe there is a setting that limits CPU usage, possibly “Power saver” in the image below.

from [All Windows 10 Power Options Explained](https://geekflare.com/windows-power-options/)

 ![image](https://global.discourse-cdn.com/julialang/original/3X/d/0/d0098f87c1f0bc6e1aaaef8c3a5e2ca9f00c6aab.jpeg)