Nested parallelization in Julia?

Hello,

I was curious to know if there has been some thought regarding nested parallelization in julia. For example, a typical parallelizable routine with a for loop inside may look like

function my_parallel_function()
 @distributed () for i in 1:I
  #do something 
   for j in 1:J 
    #do something else
   end
 end
end

In cases where both | I | and |J| are large it may be useful to parallelize the inner loop within the already parallelized outer-loop. I have written parallelized code over a collection of iterators but I am wondering if this was somewhere in the works too.

in 1.3, you can nest Threads.@threads which can help here.

1 Like

nice, that’s pretty cool. We only have 1.3.0-rc4 now though, but not the stable version yet – is that right?

that’s correct, you can try it now (I think rc4 is really close to release, which should happen within the year), if you don’t have mission-critical stability requirement.

Yes, will do and I also just dug up this link (https://julialang.org/blog/2019/07/multithreading) which is useful and gives examples for what you mentioned in your first comment. Posting the link here in case some may have missed this blog post.