I have a loop which I am currently multi-threading using the :static scheduler: threadcache = [create_cache() for i in 1:Threads.nthreads()] Threads.@threads :static for i in 1:... data = threadcache[Threads.threadid()] result = compute_stuff(..., data) #data will be mutated in this functio…

See: Home · ChunkSplitters.jl And here there is a discussion exactly about load balancing in this context: Parallel load balancing · JuliaNotes.jl But basically, you can do this: using ChunkSplitters function run(;nchunks=Threads.nthreads()) threadcache = [create_cache() for i in 1:nchunks] …

Something like this? threadcache = [create_cache() for i in 1:Threads.nthreads()] idx = Threads.Atomic{Int}(N) @sync for thr in eachindex(threadcache) Threads.@spawn begin data = threadcache[$thr] while (i = Threads.atomic_sub!(idx, 1)) > 0 result = mutate(i, data) …

Thanks! But it feels like that this is more complicated than what should be needed? :sweat_smile: Refining my google search a bit, I found this post which uses Channels instead. I have never used channels before, but it looks a bit simpler. I will try it out It feels like my problem/questions woul…

This is great, thank you. I will try ChunkSplitters.jl and see how it works in my code!

Multi-threaded code with :dynamic scheduler and mutable data

General Usage

Salmon April 26, 2023, 9:27am 4

regarding the use of Channels, let me link to this Post, which has a nice code example that pretty much directly applies to what you are trying to do here

Topic		Replies	Views
Multithreading with dynamic scheduler Performance	10	1014	August 19, 2020
Multithreading workflow - alternative for :static scheduling General Usage multithreading	2	423	September 15, 2022
Behavior of `Threads.@threads for` loop General Usage multithreading	30	5350	February 14, 2022
Multi-threading appears to be single thread when some threads cost much more time than the others? General Usage question	13	827	July 10, 2023
Multithreading with shared memory caches Performance question , performance , multithreading	27	2158	June 13, 2023

Multi-threaded code with :dynamic scheduler and mutable data

Related topics