# Julia Threads.@threads slower than single thread performance

**URL:** <https://discourse.julialang.org/t/julia-threads-threads-slower-than-single-thread-performance/46823>\
**Category:** Performance\
**Tags:** multithreading, pde\
**Created:** [September 18, 2020, 5:04am UTC](https://discourse.julialang.org/t/julia-threads-threads-slower-than-single-thread-performance/46823 "2020-09-18T05:04:47Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![cricci](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cricci/32/25160_2.png) [@cricci](https://discourse.julialang.org/u/cricci)\
**Post date:** [September 18, 2020, 5:04am UTC](https://discourse.julialang.org/t/julia-threads-threads-slower-than-single-thread-performance/46823/1 "2020-09-18T05:04:47Z")

</div>

I am trying to solve numerically the heat equation in 1d:

[![enter image description here](https://global.discourse-cdn.com/julialang/original/3X/1/1/111ccfb4515b19078be945205a78eab4d4515951.gif)](https://i.stack.imgur.com/8GbwB.gif)

I am using finite differences and I have some trouble using the @threads instruction in Julia. In particular below there are two version of the same code: the first one is single thread while the other uses @threads (they are identical apart from the @thread instruction)

```julia
function heatSecLoop(;T::Float64)

    println("start")
    L = 1
    ν = 0.5
    Δt = 1e-6
    Δx = 1e-3

    Nt = ceil(Int, T/Δt )
    Nx = ceil(Int,L/Δx + 2)
    u = zeros(Nx)    
    u[round(Int,Nx/2)] = 1
    u_old = zeros(Nx)

    println("starting loop")
    for t=1:Nt-1
        u_old .= u
        for i=2:Nx-1
            u[i] = u_old[i] + ν * Δt/(Δx^2)*(u_old[i-1]-2*u_old[i] + u_old[i+1])
        end

        if t % round(Int,Nt/10) == 0
            println("time = " * string(t*Δt) )
        end
    end
    println("done")
    return u
end

function heatParLoop(;T::Float64)

    println("start")
    L = 1
    ν = 0.5
    Δt = 1e-6
    Δx = 1e-3

    Nt = ceil(Int, T/Δt )
    Nx = ceil(Int,L/Δx + 2)
    u = zeros(Nx)    
    u[round(Int,Nx/2)] = 1
    u_old = zeros(Nx)
    
    println("starting loop")
    for t=1:Nt-1
        u_old .= u
        Threads.@threads for i=2:Nx-1
             u[i] = u_old[i] + ν * Δt/(Δx^2)*(u_old[i-1]-2*u_old[i] + u_old[i+1])
        end

        if t % round(Int,Nt/10) == 0
            println("time = " * string(t*Δt) )
        end
    end
    println("done")
    return u
end

```

The issue is that the sequential one is faster than the multi-thread one. Here are the timing (after one run to compile)

```julia
julia> Threads.nthreads()
2

julia> @time heatParLoop(T=1.0)
start
starting loop
time = 0.1
time = 0.2
time = 0.3
time = 0.4
time = 0.5
time = 0.6
time = 0.7
time = 0.8
time = 0.9
done
  5.417182 seconds (12.14 M allocations: 9.125 GiB, 6.59% gc time)

julia> @time heatSecLoop(T=1.0)
start
starting loop
time = 0.1
time = 0.2
time = 0.3
time = 0.4
time = 0.5
time = 0.6
time = 0.7
time = 0.8
time = 0.9
done
  3.892801 seconds (1.00 M allocations: 7.629 GiB, 8.06% gc time)

```

Of course the heat equation is just an example for a more intricate problem. I also tried using other libraries such as SharedArrays together with Distributed but with worse results.

Any help is appreciated.

---

<div class="post-metadata">

**Author:** ![rveltz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rveltz/32/2707_2.png) [@rveltz](https://discourse.julialang.org/u/rveltz)\
**Post date:** [September 18, 2020, 6:07am UTC](https://discourse.julialang.org/t/julia-threads-threads-slower-than-single-thread-performance/46823/2 "2020-09-18T06:07:41Z")

</div>

1000 unknowns is probably to small. Your problem is not limited by computation but by the speed of access to memory. Hence, threads will not help here. You can try perhaps `@avx` from LoopVectorization.

---

<div class="post-metadata">

**Author:** ![rveltz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rveltz/32/2707_2.png) [@rveltz](https://discourse.julialang.org/u/rveltz)\
**Post date:** [September 18, 2020, 6:24am UTC](https://discourse.julialang.org/t/julia-threads-threads-slower-than-single-thread-performance/46823/3 "2020-09-18T06:24:17Z")

</div>

> [@cricci](#):
>
> `i.-1`

you should remove the dot, although it should have no impact

---

<div class="post-metadata">

**Author:** ![cricci](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cricci/32/25160_2.png) [@cricci](https://discourse.julialang.org/u/cricci)\
**Post date:** [September 18, 2020, 6:43am UTC](https://discourse.julialang.org/t/julia-threads-threads-slower-than-single-thread-performance/46823/4 "2020-09-18T06:43:12Z")

</div>

Thanks! will do

---

<div class="post-metadata">

**Author:** ![lungben](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lungben/32/12314_2.png) [@lungben](https://discourse.julialang.org/u/lungben)\
**Post date:** [September 18, 2020, 6:56am UTC](https://discourse.julialang.org/t/julia-threads-threads-slower-than-single-thread-performance/46823/5 "2020-09-18T06:56:01Z")

</div>

In general, spawning threads takes some time, therefore it is usually better to write `@threads` on your outermost loop. This way, you save thread spawning time, but still all your CPU cores will be occupied.

---

<div class="post-metadata">

**Author:** ![cricci](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cricci/32/25160_2.png) [@cricci](https://discourse.julialang.org/u/cricci)\
**Post date:** [September 18, 2020, 6:58am UTC](https://discourse.julialang.org/t/julia-threads-threads-slower-than-single-thread-performance/46823/6 "2020-09-18T06:58:01Z")

</div>

@rveltz  
I also tried \Delta t = 1e-8 and \Delta x = 1e-4 with similar results for what concern @threads

```julia
julia> @time heatParLoop(T=0.01)
start
starting loop
time = 0.001
time = 0.002
time = 0.003
time = 0.004
time = 0.005
time = 0.006
time = 0.007
time = 0.008
time = 0.009000000000000001
done
 31.064373 seconds (11.19 M allocations: 1.496 GiB, 0.72% gc time)

julia> @time heatSecLoop(T=0.01)
start
starting loop
time = 0.001
time = 0.002
time = 0.003
time = 0.004
time = 0.005
time = 0.006
time = 0.007
time = 0.008
time = 0.009000000000000001
done
 21.752236 seconds (109 allocations: 162.859 KiB)

```

However the package LoopVectorization and @avx it’s awesome

```julia
using LoopVectorization
function heatVecLoop(;T::Float64)

    println("start")
    L = 1
    ν = 0.5
    Δt = 1e-8
    Δx = 1e-4

    Nt = ceil(Int, T/Δt )
    Nx = ceil(Int,L/Δx + 2)
    u = zeros(Nx)    
    u[round(Int,Nx/2)] = 1
    u_old = zeros(Nx)
    
    println("starting loop")
    for t=1:Nt-1
        u_old .= u
        @avx for i=2:Nx-1
             u[i] = u_old[i] + ν * Δt/(Δx^2)*(u_old[i-1]-2*u_old[i] + u_old[i+1])
        end

        if t % round(Int,Nt/10) == 0
            println("time = " * string(t*Δt) )
        end
    end
    println("done")
    return u
end
julia> @time heatVecLoop(T=0.01)
start
starting loop
time = 0.001
time = 0.002
time = 0.003
time = 0.004
time = 0.005
time = 0.006
time = 0.007
time = 0.008
time = 0.009000000000000001
done
  5.577220 seconds (109 allocations: 162.859 KiB)

```

Thanks!

---

<div class="post-metadata">

**Author:** ![lungben](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lungben/32/12314_2.png) [@lungben](https://discourse.julialang.org/u/lungben)\
**Post date:** [September 18, 2020, 6:59am UTC](https://discourse.julialang.org/t/julia-threads-threads-slower-than-single-thread-performance/46823/7 "2020-09-18T06:59:25Z")

</div>

> [@cricci](#):
>
> `u_old = copy(u)`

As far as I can see, this copy is not required. It causes a large amount of allocations and probably quite some execution time.

---

<div class="post-metadata">

**Author:** ![cricci](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cricci/32/25160_2.png) [@cricci](https://discourse.julialang.org/u/cricci)\
**Post date:** [September 18, 2020, 7:00am UTC](https://discourse.julialang.org/t/julia-threads-threads-slower-than-single-thread-performance/46823/8 "2020-09-18T07:00:19Z")

</div>

@lungben  
That I knew. However in this situation I don’t know if there is a way to do that, since the outer most loop cannot be executed asynchronously since each step require the previous one to execute.

However you are somewhat right about the `copy` instruction. It is strictly needed, but perhaps is better by assigning with `.=` into a preallcated variable to save time. However all the benchmarks show similar results as before  
(edited original code to avoid copy)

---

<div class="post-metadata">

**Author:** ![cricci](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cricci/32/25160_2.png) [@cricci](https://discourse.julialang.org/u/cricci)\
**Post date:** [September 18, 2020, 7:21am UTC](https://discourse.julialang.org/t/julia-threads-threads-slower-than-single-thread-performance/46823/9 "2020-09-18T07:21:08Z")

</div>

FYI, the results are still kind of the same even by taking

```julia
 Δt = 1e-10
 Δx = 1e-5

```

```julia
@time heatSecLoop(T=0.00001)
 26.710780 seconds (109 allocations: 1.532 MiB)
@time heatParLoop(T=0.00001)
 23.646293 seconds (1.20 M allocations: 158.215 MiB, 0.18% gc time)
@time heatVecLoop(T=0.00001)
  9.190979 seconds (109 allocations: 1.532 MiB)

```

now the @thread version is slightly faster than the single threaded one (not by much) and the version with LoopVectorization still crushes them.

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [September 18, 2020, 9:54am UTC](https://discourse.julialang.org/t/julia-threads-threads-slower-than-single-thread-performance/46823/10 "2020-09-18T09:54:50Z")

</div>

I get better performance from this:

```julia
function heatFMLoop(;T::Float64)
    println("start")
    L = 1
    ν = 0.5
    Δt = 1e-8
    Δx = 1e-4
    Nt = ceil(Int, T/Δt )
    Nx = ceil(Int,L/Δx + 2)
    u = zeros(Nx)
    u[round(Int,Nx/2)] = 1
    u_old = similar(u)
    println("starting loop")
    for t=1:Nt-1
        u_old, u = u, u_old
        @inbounds @fastmath for i=2:Nx-1
             u[i] = u_old[i] + ν * Δt/(Δx^2)*(u_old[i-1]-2*u_old[i] + u_old[i+1])
        end
        if t % round(Int,Nt/10) == 0
            println("time = " * string(t*Δt) )
        end
    end
    println("done")
    return u
end

```

Results:

```julia
julia> @time heatVecLoop(T = 0.01); # after modifying it similarly
start
starting loop
time = 0.001
time = 0.002
time = 0.003
time = 0.004
time = 0.005
time = 0.006
time = 0.007
time = 0.008
time = 0.009000000000000001
done
  1.874391 seconds (104 allocations: 162.641 KiB)

julia> @time heatFMLoop(T = 0.01);
start
starting loop
time = 0.001
time = 0.002
time = 0.003
time = 0.004
time = 0.005
time = 0.006
time = 0.007
time = 0.008
time = 0.009000000000000001
done
  1.756562 seconds (104 allocations: 162.641 KiB)

```

I’ll have to look into what LoopVectorization is doing wrong here.

---

<div class="post-metadata">

**Author:** ![cricci](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cricci/32/25160_2.png) [@cricci](https://discourse.julialang.org/u/cricci)\
**Post date:** [September 27, 2020, 4:17am UTC](https://discourse.julialang.org/t/julia-threads-threads-slower-than-single-thread-performance/46823/11 "2020-09-27T04:17:41Z")

</div>

FYI, i tried the same approach with the same problem (heat equation) but in 2D. Same exact results if not that in this case the use of @threads is even more dreadful than in 1D (also here I think is simply related to memory problem).  
At present, I haven’t understood yet how to write efficient parallel code to solve such type of PDEs with Julia (or any other time-evolution process where you cannot parallelize the outer most loop)

---

<div class="post-metadata">

**Author:** ![cocoa1231](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cocoa1231/32/22215_2.png) [@cocoa1231](https://discourse.julialang.org/u/cocoa1231)\
**Post date:** [April 24, 2023, 8:57am UTC](https://discourse.julialang.org/t/julia-threads-threads-slower-than-single-thread-performance/46823/12 "2023-04-24T08:57:11Z")

</div>

Hello, my code is facing similar issues. I’m iterating over a `Tables.namedtupleiterator` (after calling `collect` on the iterator) whose rows are the bins of a histogram and the multi threaded code seems to perform worse than the single threaded version. I’ve added `Threads.@threads` on the outer most loop. Has anybody had success with this so far?