# Multithreading with seperate caches for each thread?

**URL:** <https://discourse.julialang.org/t/multithreading-with-seperate-caches-for-each-thread/116441>\
**Category:** General Usage\
**Created:** [June 30, 2024, 7:48pm UTC](https://discourse.julialang.org/t/multithreading-with-seperate-caches-for-each-thread/116441 "2024-06-30T19:48:53Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![recklurker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/recklurker/32/205292_2.png) [@recklurker](https://discourse.julialang.org/u/recklurker)\
**Post date:** [June 30, 2024, 7:48pm UTC](https://discourse.julialang.org/t/multithreading-with-seperate-caches-for-each-thread/116441/1 "2024-06-30T19:48:53Z")

</div>

Hi everyone,  
I am writing a finite element solver. Since the solver takes a lot of time on a single core, I am trying to parallelize it. The solver method has several functional calls which share some cache. This has been done to keep the solver from allocating and to reduce time spent in garbage collection.

For parallelizing this, I am defining an array containing a cache struct for each thread. To prevent any race conditions, I am also tracking the cache used by a thread by maintaining a free\_cache boolean array.

The code will look something like this:

```julia
chunks = Iterators.partition(eachindex(A), div(length(A), Threads.nthreads())
cache = [Cache{Float64, Int64}() for _ in Threads.nthreads()]
free_cache = fill(Bool(1), Threads.nthreads())

tasks = map(chunks) do chunk
           @spawn do_something(A, chunk, cache, free_cache)
        end 
result = maximum(fetch.(tasks))

```

And the function’s defintion looks like:

```julia
function do_something(A, chunk, cache, free_cache)
   free_idx = findfirst(!=(0), free_cache)
   free_cache[free_idx] = 0 
   ### SOME COMPUTATION using cache[free_idx] ####
   free_cache[free_idx] = 1
   return result
end

```

Upon testing, this approach produces the correct result. I wanted to ask if this a good practice and if there are any issues that can occur with this approach?

---

<div class="post-metadata">

**Author:** ![Satvik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/satvik/32/20486_2.png) [@Satvik](https://discourse.julialang.org/u/Satvik)\
**Post date:** [June 30, 2024, 8:13pm UTC](https://discourse.julialang.org/t/multithreading-with-seperate-caches-for-each-thread/116441/2 "2024-06-30T20:13:25Z")

</div>

I believe you have a potential race condition if two different tasks finish ` free_idx = findfirst(!=(0), free_cache)` before running the next line.

One way to resolve would be to specifically assign a cache to a task. For example, you could do something like

```julia
tasks = map(chunks) do chunk
           free_cache_for_task = ....
           @spawn do_something(A, chunk, cache, free_cache_for_task)
        end 

```

Basically you want to minimize the amount of shared memory that gets passed to `@spawn`.  
For more detail, see [PSA: Thread-local state is no longer recommended](https://julialang.org/blog/2023/07/PSA-dont-use-threadid/#better_fix_work_directly_with_tasks)

---

<div class="post-metadata">

**Author:** ![recklurker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/recklurker/32/205292_2.png) [@recklurker](https://discourse.julialang.org/u/recklurker)\
**Post date:** [June 30, 2024, 8:28pm UTC](https://discourse.julialang.org/t/multithreading-with-seperate-caches-for-each-thread/116441/3 "2024-06-30T20:28:07Z")

</div>

Got it! Thanks. But there still remains the problem of allocating a lot.  
Is it not possible to allocate once and use the same cache throughout?

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [June 30, 2024, 8:32pm UTC](https://discourse.julialang.org/t/multithreading-with-seperate-caches-for-each-thread/116441/4 "2024-06-30T20:32:30Z")

</div>

You mean you want to reuse the caches later again with a new set of tasks? You could e.g. put the Caches into a `Channel` and each task just `take!`s one cache from the channel at the start instead of instantiating a new one. After finishing the task `put!`s it back into the `Channel`. A channel is thread-safe.

---

<div class="post-metadata">

**Author:** ![Salmon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/salmon/32/22968_2.png) [@Salmon](https://discourse.julialang.org/u/Salmon)\
**Post date:** [June 30, 2024, 8:39pm UTC](https://discourse.julialang.org/t/multithreading-with-seperate-caches-for-each-thread/116441/5 "2024-06-30T20:39:00Z")

</div>

I like ChunkSplitters.jl for this kind of thing

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [June 30, 2024, 8:55pm UTC](https://discourse.julialang.org/t/multithreading-with-seperate-caches-for-each-thread/116441/6 "2024-06-30T20:55:48Z")

</div>

> **[Thread-Safe Storage · OhMyThreads.jl](https://juliafolds2.github.io/OhMyThreads.jl/stable/literate/tls/tls/)**
>
> Documentation for OhMyThreads.jl.

---

<div class="post-metadata">

**Author:** ![sgaure](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sgaure/32/14779_2.png) [@sgaure](https://discourse.julialang.org/u/sgaure)\
**Post date:** [June 30, 2024, 9:01pm UTC](https://discourse.julialang.org/t/multithreading-with-seperate-caches-for-each-thread/116441/7 "2024-06-30T21:01:16Z")

</div>

You can use the task-local-storage. I use a macro (or two) to keep e.g. vectors around in each task, i.e. typically a spawned thread:

```julia
macro tlscache(type, makeit)
    sym = Expr(:quote, gensym("tls"))
    quote
        get!(() -> $(esc(makeit)), task_local_storage(), ($sym,$(esc(type))))::$(esc(type))
    end
end

macro tlscache(type)
    sym = Expr(:quote, gensym("tls"))
    quote
        get!(() -> $(esc(type))(), task_local_storage(), ($sym,$(esc(type))))::$(esc(type))
    end
end

```

It’s used like this:

```julia
v = @tlscache Vector{Int}
resize!(v, 23)
w = @tlscache Matrix{Float64} zeros(15,20)

```

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [June 30, 2024, 9:04pm UTC](https://discourse.julialang.org/t/multithreading-with-seperate-caches-for-each-thread/116441/8 "2024-06-30T21:04:25Z")

</div>

An example of the same idea but without macros can be seen here: [PSA: Thread-local state is no longer recommended; Common misconceptions about threadid() and nthreads() - #26 by Mason](https://discourse.julialang.org/t/psa-thread-local-state-is-no-longer-recommended-common-misconceptions-about-threadid-and-nthreads/101274/26)