Nested parallelization in Julia?

General Usage
1

Hello,

I was curious to know if there has been some thought regarding nested parallelization in julia. For example, a typical parallelizable routine with a for loop inside may look like

function my_parallel_function()
 @distributed () for i in 1:I
  #do something 
   for j in 1:J 
    #do something else
   end
 end
end

In cases where both | I | and |J| are large it may be useful to parallelize the inner loop within the already parallelized outer-loop. I have written parallelized code over a collection of iterators but I am wondering if this was somewhere in the works too.

2

in 1.3, you can nest Threads.@threads which can help here.

1 Like
3

nice, that’s pretty cool. We only have 1.3.0-rc4 now though, but not the stable version yet – is that right?

4

that’s correct, you can try it now (I think rc4 is really close to release, which should happen within the year), if you don’t have mission-critical stability requirement.

5

Yes, will do and I also just dug up this link (Announcing composable multi-threaded parallelism in Julia) which is useful and gives examples for what you mentioned in your first comment. Posting the link here in case some may have missed this blog post.