# PSA: Thread-local state is no longer recommended; Common misconceptions about threadid() and nthreads()

**URL:** <https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274>\
**Category:** General Usage\
**Tags:** announcement, multithreading\
**Created:** [July 6, 2023, 6:12pm UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274 "2023-07-06T18:12:08Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![Chong\_Wang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chong_wang/32/20307_2.png) [@Chong\_Wang](https://discourse.julialang.org/u/Chong_Wang)\
**Post date:** [July 7, 2023, 11:40pm UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/21 "2023-07-07T23:40:38Z")

</div>

It looks like the same logic applies to process-local states with multiprocessing. Am I right?

Edit:

To be more precise, something like this

```plaintext
results = SharedArray{Float64}(nworkers())
pmap(f, collection) # f works on on results[myid()]

```

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [July 8, 2023, 12:11am UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/22 "2023-07-08T00:11:03Z")

</div>

> [@Palli](#):
>
> or in fact that such code is always known know to be buggy?

Only if using `threadid()` to update a shared buffer. If `threadid()` was deprecated there would not be anything wrong with `@threads`, people would be forced to use the other patterns to index shared arrays. The problem is that using it is very tempting.

---

<div class="post-metadata">

**Author:** ![ianshmean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ianshmean/32/216042_2.png) [@ianshmean](https://discourse.julialang.org/u/ianshmean)\
**Post date:** [July 8, 2023, 1:01am UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/23 "2023-07-08T01:01:08Z")

</div>

Alongside this blogpost we also tried to improve the docs and guide against the bad patterns. It’d be good to see whether people think the newly updated 1.9.2 docs are clear

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [July 8, 2023, 1:37am UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/24 "2023-07-08T01:37:07Z")

</div>

> [@Palli](#):
>
> @elrod Polyester.jl has been on my radar as a replacement for `@threads`. Since it’s not on the list of (apparently) approved packages listed there, is it then know to be buggy too? Or if it’s actually known to (_always_) produce correct code when it’s used, then you may want to add it to the list in the post. It would be good to have a full block-list of known threading packages to avoid and/or a full allow-list, e.g. are all the dependencies ok, such as ThreadingUtilities.jl?

Polyester’s threads are sticky, like `@threads :static`.  
It also supports `threadlocal=` for defining thread-local state, without needing `Threads.@threadid` (which you should still avoid).  
Unfortunately, it is type unstable and allocates memory.

I give some ideas for how it can be improved if anyone wants to give it a try (good first issue):

> <https://github.com/JuliaSIMD/Polyester.jl/issues/112>
>
> The \`threadlocal\` which appears outside the \`@batch\` is not inferrable, even if …the type is given in the macro.
> 
> The following code (inspired from the README)
> \`\`\`julia
> function f()
> @batch threadlocal=rand(10:99)::Int for i in 0:9
> threadlocal += 1
> end
> sum(threadlocal)
> end
> 
> @code\_warntype f()
> \`\`\`
> tells me that the content of \`threadlocal\` and the return type of the function have both type \`Any\`.

Note that LoopVectorization.jl handles threadlocal state without allocations.

We could use something like that approach, at least for many of the common cases.

```julia
julia> using LoopVectorization, LinearAlgebra, LinuxPerf

julia> BLAS.set_num_threads(@show(Threads.nthreads()));
Threads.nthreads() = 4

julia> x = rand(2^20);

julia> function dottturbo(x,y)
           s = zero(Base.promote_eltype(x,y))
           @tturbo for i = eachindex(x,y)
               s += x[i]*y[i]
           end
           return s
       end
dottturbo (generic function with 1 method)

julia> foreachf!(f::F, N, args::Vararg{<:Any,A}) where {F,A} = foreach(_ -> Base.donotdelete(f(args...)), Base.OneTo(N))
foreachf! (generic function with 1 method)

julia> @pstats "cpu-cycles,(instructions,branch-instructions,branch-misses),(task-clock,context-switches,cpu-migrations,page-faults),(L1-dcache-load-misses,L1-dcache-loads,L1-icache-load-misses),(dTLB-load-misses,dTLB-loads),(iTLB-load-misses,iTLB-loads)" begin
       foreachf!(dot, 100_000, x, x)
       end
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
╶ cpu-cycles 1.23e+11 60.0% # 3.6 cycles per ns
┌ instructions 5.10e+10 60.0% # 0.4 insns per cycle
│ branch-instructions 5.69e+09 60.0% # 11.2% of insns
└ branch-misses 2.40e+06 60.0% # 0.0% of branch insns
┌ task-clock 3.40e+10 100.0% # 34.0 s
│ context-switches 0.00e+00 100.0%
│ cpu-migrations 0.00e+00 100.0%
└ page-faults 0.00e+00 100.0%
┌ L1-dcache-load-misses 1.32e+10 20.0% # 45.0% of dcache loads
│ L1-dcache-loads 2.93e+10 20.0%
└ L1-icache-load-misses 1.46e+07 20.0%
┌ dTLB-load-misses 5.65e+02 20.0% # 0.0% of dTLB loads
└ dTLB-loads 2.92e+10 20.0%
┌ iTLB-load-misses 1.08e+05 40.0% # 30.9% of iTLB loads
└ iTLB-loads 3.49e+05 40.0%
                  aggregated from 4 threads
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

julia> @pstats "cpu-cycles,(instructions,branch-instructions,branch-misses),(task-clock,context-switches,cpu-migrations,page-faults),(L1-dcache-load-misses,L1-dcache-loads,L1-icache-load-misses),(dTLB-load-misses,dTLB-loads),(iTLB-load-misses,iTLB-loads)" begin
       foreachf!(dottturbo, 100_000, x, x)
       end
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
╶ cpu-cycles 1.23e+11 60.0% # 3.6 cycles per ns
┌ instructions 3.64e+10 60.0% # 0.3 insns per cycle
│ branch-instructions 3.38e+09 60.0% # 9.3% of insns
└ branch-misses 1.07e+06 60.0% # 0.0% of branch insns
┌ task-clock 3.38e+10 100.0% # 33.8 s
│ context-switches 0.00e+00 100.0%
│ cpu-migrations 0.00e+00 100.0%
└ page-faults 0.00e+00 100.0%
┌ L1-dcache-load-misses 1.31e+10 20.0% # 49.9% of dcache loads
│ L1-dcache-loads 2.63e+10 20.0%
└ L1-icache-load-misses 8.40e+05 20.0%
┌ dTLB-load-misses 7.50e+01 20.0% # 0.0% of dTLB loads
└ dTLB-loads 2.63e+10 20.0%
┌ iTLB-load-misses 1.08e+02 40.0% # 0.4% of iTLB loads
└ iTLB-loads 2.40e+04 40.0%
                  aggregated from 4 threads
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

julia> @btime dot($x, $x)
  83.491 μs (0 allocations: 0 bytes)
349742.18307257013

julia> @btime dottturbo($x, $x)
  82.378 μs (0 allocations: 0 bytes)
349742.18307257013

```

LV seems to ramp up thread use much more slowly though, which seems to hurt performance if you have access to more threads.  
That could also use some more work.  
I.e., LV should perhaps consider cache sizes when deciding how many threads to allocate.  
But that’d probably be more relevant for micro benchmarks, where you have a chance of holding the memory in cache, while in “real workloads”, it’s less likely the memory will be sitting in the correct core-local caches before the function gets called…  
I find it unlikely that real workloads are bottlenecked by code like the above example; they’re hopefully able to parallelize on a more grandular level, so they can fuse the dot product with some other operations.

Or, taking the [ChunkSplitters](https://m3g.github.io/ChunkSplitters.jl/stable/)’ example:

```julia
julia> using LoopVectorization

julia> using ChunkSplitters: chunks

julia> function sum_tturbo(f::F, x) where {F}
           s = zero(eltype(x))
           @tturbo for i = eachindex(x)
               s += f(x[i])
           end
           s
       end
sum_tturbo (generic function with 1 method)

julia> function sum_parallel(f, x; nchunks=Threads.nthreads())
           s = fill(zero(eltype(x)), nchunks)
           Threads.@threads for ichunk in 1:nchunks
               for i in chunks(x, ichunk, nchunks)
                   s[ichunk] += f(x[i])
               end
           end
           return sum(s)
       end
sum_parallel (generic function with 1 method)

julia> x = rand(10^7);

julia> Threads.nthreads()
36

julia> @btime sum(x -> log(x)^7, $x)
  157.715 ms (0 allocations: 0 bytes)
-4.944530243712232e10

julia> @btime sum_parallel(x -> log(x)^7, $x; nchunks=4)
  94.041 ms (206 allocations: 20.30 KiB)
-4.9445302437075905e10

julia> @btime sum_parallel(x -> log(x)^7, $x; nchunks=12)
  70.799 ms (209 allocations: 20.45 KiB)
-4.944530243710959e10

julia> @btime sum_parallel(x -> log(x)^7, $x; nchunks=64)
  29.700 ms (219 allocations: 21.17 KiB)
-4.944530243712026e10

julia> @btime sum_tturbo(x->log(x)^7, $x)
  908.156 μs (0 allocations: 0 bytes)
-4.944530243712195e10

```

`@tturbo` is 32x faster with 0 allocations 😉

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [July 8, 2023, 4:34am UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/25 "2023-07-08T04:34:49Z")

</div>

I’m confused where this leaves [`task_local_storage`](https://docs.julialang.org/en/v1/base/parallel/#Base.task_local_storage-Tuple%7BAny%7D), “task” as opposed to “thread”. As far as I can tell, that is still working, not deprecated, API.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [July 8, 2023, 4:58am UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/26 "2023-07-08T04:58:08Z")

</div>

`task_local_storage` isn’t going anywhere. It’s just not particularly useful or nice for implementing reductions.

For an example of something that `task_local_storage` does well in the context of `@threads`, here’s some code @GunnarFarneback shared on Zulip:

```julia
Threads.@threads for j in axes(x, 2)
    temp = get!(() -> Vector{Int}(undef, N), task_local_storage(), :temp)::Vector{Int}
    # fill in temp with j-dependent data
    for i in axes(x, 1)
        y[i, j] = # some computation with x, i, j, temp
    end
end

```

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [July 8, 2023, 5:09am UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/27 "2023-07-08T05:09:04Z")

</div>

> [@Palli](#):
>
> Thanks for writing this up. I and many programmers know that (naked) threads are dangerous, so I was really hoping Julia did things right with `@threads for` loops. Is my understanding correct, you should always suspect code using it to be buggy (unless it’s using the newer `@threads :static`, and then always ok?), or in fact that such code is always known know to be buggy?

No, there are many many uses of `@threads` that are totally fine. What is suspect is code which is using `threadid()` in a multithreaded context, especially code which is indexing into some buffer using that function.

Additionally, there is _advice_ that it’s just generally not a good idea to create a mutable buffer outside of a `@threads` block, and then read from and write to that buffer inside the `@threads` block. This can be done correctly, but one **does** need to be careful when doing so because it is an easy thing to do wrong.

> [@Palli](#):
>
> @elrod Polyester.jl has been on my radar as a replacement for `@threads`. Since it’s not on the list of (apparently) approved packages listed there

Just to clarify why Polyester.jl wasn’t mentioned, that’s because it is not really an active participant in Julia’s regular multithreaded scheduling system, it’s really just a rather separate system for writing multithreaded code, so we decided it wasn’t really within the scope of that blogpost.

> [@Palli](#):
>
> That’s too bad, since the threading API is no longer marked experimental and we can’t then get rid of it; until 2.0. I’m not sure if we can fix already-written code in 2.0, by e.g. making `:static` implied then? What would your ideal API be like, is there one already available in some package, and what would be your proposal for Base, to mitigate things in 1.x?

We can’t get rid of `Base.Threads` but that’s fine. That module can just stay as a set of low level primitives for multithreading, and then we could start working on a new standard library or sub-module of base that has good high-level parallel functions like `map`, `reduce`, `filter`, etc.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [July 8, 2023, 5:26am UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/28 "2023-07-08T05:26:20Z")

</div>

> [@Chong\_Wang](#):
>
> It looks like the same logic applies to process-local states with multiprocessing. Am I right?
> 
> Edit:
> 
> To be more precise, something like this
> 
> ```julia
> results = SharedArray{Float64}(nworkers())
> pmap(f, collection) # f works on on results[myid()]
> 
> ```

To be honest I don’t know, I’m not super experienced with Distributed.jl, but maybe someone like @jpsamaroo or @vchuravy could let us know. The scheduling details of processes is pretty different to tasks, so I wouldn’t be very surprised one way or another.

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [July 8, 2023, 5:28am UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/29 "2023-07-08T05:28:08Z")

</div>

Is it reasonable to have `threadid()` return a special `Integer` subtype instead of an `Int`, and deprecate indexing using that type (or warn againt this)? This might increase the visibility of this message, and urge packages to change.

---

<div class="post-metadata">

**Author:** ![simsurace](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simsurace/32/30216_2.png) [@simsurace](https://discourse.julialang.org/u/simsurace)\
**Post date:** [July 8, 2023, 7:59am UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/30 "2023-07-08T07:59:05Z")

</div>

That‘s a nice idea I haven‘t heard before. It clearly delineates the problematic indexing, whereas indexing via loop index is fine and not affected.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [July 8, 2023, 8:30am UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/31 "2023-07-08T08:30:28Z")

</div>

> [@Chong\_Wang](#):
>
> It looks like the same logic applies to process-local states with multiprocessing. Am I right?

Conceptually, yes I’d have to agree 🤔 Whether the parallelism in question is threading or parallel processing, the race conditions that are the problem are a property of the parallel algorithm, not the means of parallelization.

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [July 8, 2023, 9:00am UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/32 "2023-07-08T09:00:26Z")

</div>

I’m against it. If one has sticky tasks using `threadid()` for indexing is just fine.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [July 8, 2023, 11:08am UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/33 "2023-07-08T11:08:42Z")

</div>

Anyway the change would be breaking, but it wouldn’t be bad having something like `threadid(warn=false)` to not throw a warning vs a default warning version.

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [July 8, 2023, 11:09am UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/34 "2023-07-08T11:09:44Z")

</div>

Within a process `myid()` is stable, whereas within a task `threadid()` might not be.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [July 8, 2023, 5:41pm UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/35 "2023-07-08T17:41:23Z")

</div>

And are all the ids unique? Ie. can two separate processes have the same id? Id assume not, right?

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [July 8, 2023, 6:47pm UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/36 "2023-07-08T18:47:15Z")

</div>

> [@Mason](#):
>
> can two separate processes have the same id?

On the same machine, no, if you’re using a cluster then processes on different machines could easily have the same PID and if they use that PID to identify themselves when handing data back to some central coordinator there could be conflict/collisions.

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [July 8, 2023, 7:52pm UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/37 "2023-07-08T19:52:07Z")

</div>

Julia worker processes have unique IDs (independent on which machine they run on).

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [July 8, 2023, 9:18pm UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/38 "2023-07-08T21:18:44Z")

</div>

Why is that I cannot use `@turbo` in the inner loop, like this?

```julia
julia> function sum_parallel(f, x; nchunks=Threads.nthreads())
           s = fill(zero(eltype(x)), nchunks)
           Threads.@threads for (irange, ichunk) in chunks(x, nchunks)
               @turbo for i in irange
                   s[ichunk] += f(x[i])
               end
           end
           return sum(s)
       end
ERROR: LoadError: KeyError: key Symbol("###RHS###6###") not found

```

`irange` is regular `UnitRange{Int64}`.

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [July 8, 2023, 11:55pm UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/39 "2023-07-08T23:55:29Z")

</div>

It doesn’t like `s[ichunk]`. If you hoist the load and store out of the loop, it should work.

---

<div class="post-metadata">

**Author:** ![Chong\_Wang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chong_wang/32/20307_2.png) [@Chong\_Wang](https://discourse.julialang.org/u/Chong_Wang)\
**Post date:** [July 8, 2023, 11:58pm UTC](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/40 "2023-07-08T23:58:29Z")

</div>

> [@vchuravy](#):
>
> Within a process `myid()` is stable, whereas within a task `threadid()` might not be.

Suppose I have 10 calculations distributed in 5 workers, and now worker 2 is running task 1. Isn’t it possible that calculation 1 yields and calculation 6 kicks in in worker 2 before task 1 finishes? That will cause a problem even if `myid()` is stable.

Or is it something like `@spawnat p expr` does not wrap `exp` in a task and will never yield before `expr` finishes?

[Previous page](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274.md?page=1)

[Next page](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274.md?page=3)