How to do distributed multithreading?

Specific Domains Julia at Scale
1

I’ve a function that uses multithreading to compute a function f with each thread value and stores the value as an array element.

 function multithread()    
     num_threads = 8
     output = Array{Float64}(num_threads)
 
     Threads.@threads for i in 1:num_threads
         output[i] = f(....)
     end
 return output
 end

I’ve cluster a of 8 CPUs with 8cores/threads available per CPU. How can I distribute multithread function across the cluster, so that function f evaluated 64 times(8 cores x 8 threads) on the cluster? How to make sure that I’m able to use multithreading while using distributed computing? Any help regarding this is much appreciated.

2

I don’t think one can have multiple threads outside of the confines of the shared memory model. In the distributed computing setting the memory is not shared.

But you could marry the distributed model with threading. That is quite common, even when harder to do than any of the two models used separately.

1 Like