Dynamic scheduling in @sync @distributed for constructions

Specific Domains Julia at Scale
1

Hello!
Maybe it is a too simple question, but I couldn’t find it in the documentation.

Constructions like

@sync @distributed for j=1:N
(body)
end

make static schedule of iterations by default. But my iterations can take different time, some are very long, some are short. In OpenMP I can easily set the schedule to dynamic before the loop to optimize performance. Do you have an analog for it?

2

Maybe you can use pmap.

https://docs.julialang.org/en/v1/manual/parallel-computing/index.html#Parallel-Map-and-Loops-1

https://docs.julialang.org/en/v1.1/stdlib/Distributed/#Distributed.pmap

3

Thanks. So it seems that there is no easy OpenMP style solution. Bad, static/dynamic schedule seems to be a very simple and useful option.

4

Could you explain more why pmap is not suitable?

Is it the syntax?
Do you want to dynamically switch between static/dynamic load scheduling?

Could you use something like:

pmap(1:N) do j
   (body)
end
5

It’s suitable but makes very simple things more complicated. I want my loop to start from the word “for” because it’s simple.

6

Ha, maybe some clever person can write a macro @pfor to sweeten your syntax.

7

=) May be, but it seems to me that Julia creators are inventing the bicycle in this area. People do not come to Julia from nowhere, and have experience with such old and common things like MPI or OpenMP.
I think problem is also that Julia documentation is very poor, with so few examples. Like now I still don’t get why you need “@sync @distributed for” and “pmap”. Which of this function will vanish in Julia 2.0?