What is currently the best way to handle task/thread-local caches/buffers that need to be accessed efficiently from functions called from a Threads.@threads for loop? The background is as follows. As described in the recent blog post PSA: Thread-local state is no longer recommended , the following p…

[image] How to correctly design multi-thread cache that's transparent to user iteration? Performance Runnable code and benchmark at this gist . This is a widespread pattern for columnar-storage access, usually each column is too large to fit in ram, so it’s chu…

[image] ranocha: Do you have any recommendations for this? You can actually do this by hand, but this is what Home · ChunkSplitters.jl does. Basically you would write that as: function foo(x; cache) # do something with x and cache return some_value end # Package PkgB using ChunkSplitt…

Home · ChunkSplitters.jl this is amazing, light-weight & useful is there a chunking scheme (and associated @threds primitive maybe) such that all threads process 1st chunk and then move to the next one? It’s close to :scatter but I want “sync” between moving to next chunk I guess want this because…

I’m not sure if that applies to your case, but you can control the number of threads and chunks independently. If you increase nchunks, each task will be lighter, thus by controlling the number of parallel threads being run (by the nthreads() set on Julia startup) it might be possible to tune the to…

maybe I can put it in a different way. Imagine I only want to keep one chunk in RAM (my target full array is lazily read out), and have all the threads work on it until it’s done. Is there a way to do it? btw, I can’t control the chunk size, that’s determined by the file format (my array is an abst…

Maybe you could use ChunkSplitters to partition the tasks, but not parallelizing the outer loop, but the inner iteration, something like: julia> nchunks = 20 for (task_range, ichunk) in ChunkSplitters.chunks(tasks, nchunks) # load buffer buff[ichunk] @threads for task_i…

[image] jling: maybe can provide some inspiration There wasn’t really an answer to your question in the thread you linked, was it?

thus only inspiration, depending on the problem size, you can use the task_local_storage() shown. The double-lock pattern also works btw, where you have a vector of nthreads() locks, and each task take the lock before working on corresponding cache index

Thanks for all your answers/suggestions/comments so far! [image] lmiq: I don’t know of any solution that avoids passing cache (or at least the ichunk index) to foo as a parameter, though. That’s really the tough part for me. The problem is a little bit more complex than I sketched in my init…

Indeed, that is harder. I don’t see any way to do that safely without paying the price of a lock somewhere. Using channels is probably a reasonable alternative, unless this is a very, very, tight loop.

Task/thread-local caches/buffers

General Usage Performance

jling July 7, 2023, 12:06pm 2

maybe can provide some inspiration

Topic		Replies	Views
How to correctly design multi-thread cache that's transparent to user iteration? Performance question , hep	10	996	May 31, 2023
Writing to a shared buffer in multi-threaded for loop Performance multithreading	17	535	June 7, 2026
Pattern for managing thread local storage? General Usage question , multithreading	5	2260	May 17, 2021
Multithreading with seperate caches for each thread? General Usage	7	277	June 30, 2024
Multithreading with shared memory caches Performance question , performance , multithreading	27	2195	June 13, 2023

Task/thread-local caches/buffers

Related topics