# How to use many (non-sticky) tasks while maximizing local storage reuse?

**URL:** <https://discourse.julialang.org/t/how-to-use-many-non-sticky-tasks-while-maximizing-local-storage-reuse/100279>\
**Category:** General Usage\
**Tags:** multithreading, task\_local\_storage\
**Created:** [June 13, 2023, 11:50am UTC](https://discourse.julialang.org/t/how-to-use-many-non-sticky-tasks-while-maximizing-local-storage-reuse/100279 "2023-06-13T11:50:47Z")\
**Posts on this page:** 1\
**Showing post:** 6

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [June 13, 2023, 6:20pm UTC](https://discourse.julialang.org/t/how-to-use-many-non-sticky-tasks-while-maximizing-local-storage-reuse/100279/6 "2023-06-13T18:20:29Z")

</div>

> [@pablosanjose](#):
>
> **Question 1** : If `number_of_chunks = nthreads()`, aren’t the two approaches equivalent?
> 
> If `number_of_chunks > nthreads()` we should get some degree of load balancing with the second approach, which is nice. However we also increase the number of `Solver()` copies, which is not so nice.

The two approaches are mostly equivalent, but the problem with using `@threads :static` is that if there’s anything else multithreaded going on elsewhere in your code that you’re not aware of, it’ll destructively interfere.

E.g. if `solve!` is actually doing some multithreaded stuff interally, you could end up being reduced to worse than single threaded speeds if you use `@threads :static`, but that is not the case with `@spawn`.

> [@pablosanjose](#):
>
> **Question 2** : Is there any way we can get `number_of_chunks > nthreads()` but still use only `nthreads()` copies of the `Solver()` storage that get reused?
> 
> I think Question 2 should be possible (and optimal) if we had sticky tasks by default. We could then still use `threadid()` and get load-balanding by spawning lots of sticky tasks that digested as threads become idle. So non-sticky tasks seem like a drag in my (probably naive) view.

This should be possible if you combine a channel with the chunking approach as put forth here: [Multithreading with shared memory caches - #6 by danielwe](https://discourse.julialang.org/t/multithreading-with-shared-memory-caches/100194/6)

For your case, that’d look something like

```julia
function solve2(inputs; number_of_chunks = 20 * nthreads())
    solutions = Vector{Solution}(undef, length(inputs))
    
	chunk_size = max(1, length(inputs) ÷ number_of_chunks)
	chunks = Iterators.partition(enumerate(inputs), chunk_size)

    chunk_queue = Channel{eltype(chunks)}(Inf)
    foreach(chunk -> put!(ch, chunk), chunks)
    close(chunk_queue)
    
	@sync for _ ∈ 1:nthreads()
        @spawn begin
            solver = Solver()
            for chunk ∈ chunk_queue
                for (i, input) in chunk
                    solutions[i] = solve!(solve)
                end
            end
        end
	end
	return solutions
end

```

In this case, we only ever create `nthreads()` different tasks, but there’s by default `20` chunks per task, and each task is pulling chunks to work on from the shared `Channel`.

Taking from the channel has some overhead due to locks, so we don’t want to do it every iteration, but that’s why we store the work in chunks so that after taking a chunk, we can do a fast sequential loop over the chunk.

There may be a more elegant and more efficient way to write this, I’m not sure, I’m not an experienced user of channels or this specific pattern.

> [@pablosanjose](#):
>
> **Question 3** : What do we gain from non-sticky tasks? Can we disable them somehow so we can still use `threadid()` or would that be a bad idea for some reason?
> 
> XRef: [Multithreading, preallocation of objects, threadid() and task migration](https://discourse.julialang.org/t/multithreading-preallocation-of-objects-threadid-and-task-migration/67640)

Mostly, we gain composability. One function can do multithreading without worrying if some other function is doing multithreading. We also gain a lot of general flexibility beyond being able to just statically schedule a for loop which is all the old scheduler was really capable of.

---

_[View the full topic](https://discourse.julialang.org/t/how-to-use-many-non-sticky-tasks-while-maximizing-local-storage-reuse/100279)._