Request for comments on approach to multithreaded work

Hi,

I’m developing a multithreaded version of a piece of software and already have a working solution. I just want to check if there is a simpler way of doing it.

Basically, the problem can be reduced to this: suppose we need to complete N jobs. To do this, I have nres resources available—think of a machine in a factory (in my case it is a struct that is expensive to instantiate). Finally, there are nthrds workers that can operate those machines to complete one of the N jobs. Importantly, only one worker may operate a resource at any time.

Here’s a toy example that achieves exactly this

using Base.Threads

nthrds = nthreads() # number of threads (or workers)
nres = nthrds # number of resources on which threads can work
N = 20 # total number jobs that have to be completed

# channel containing id of the idle resources
const idle_res = Channel{Int}(nres)
foreach(i->put!(idle_res,i),1:nres) # initialize with all resources available

# function to complete a job using a given resource res
function do_work(res)
    sleep(0.5)
end

# do jobs using nthrds workers, taking and putting back idle resources
Threads.@threads for job in 1:N
    res = take!(idle_res) # take an idle resource. no other thread can now work on this resource
    do_work(res) # do job with the resource res
    println("thread ", threadid(), ": ", "completed job ", job, " using resource ", res)
    put!(idle_res, res) # mark resource as idle
end

My question is: have I missed a simpler way of achieving this in Julia?

2 Likes

Looks already pretty nice to me!

2 Likes

thanks for taking the time to review it!