Use break or continue statement with pmap

General Usage
, ,
1

Is there any alternative to continue or break when using a pmap from the distributed package? If I try the following minimal example:

using Distributed

as = [i for i in 0:0.1:10]

status = pmap(as) do a
    if a < 1
        continue
    end
    println(a)
end

I get the error: syntax: break or continue outside loop. Is there any way to obtain the same functionality with distributed computing?

2

for something like a break statement, it’s sort of hard to imagine what you would want to happen in a distributed computation (since the order in which the computations get evaluated is not predetermined).

If all you want is to be able to skip part of the computation, as you have above, then just an if statment will do:

using Distributed

as = [i for i in 0:0.1:10]

status = pmap(as) do a
    if a >= 1
        println(a)
    end
end
3 Likes
3

This has nothing in particular to do with pmap but more to do with the do construction. Since that is an anonymous function in disguise, you can do

status = pmap(as) do a
    if a < 1
        return
    end
    println(a)
end

although I wouldn’t really recommend it.

1 Like
4

Your post (in combination with @jacobadenbaum’s) reads a bit like it would abort the computation (like break). So for the clarity of anyone reading this in the future: The return does not stop the computation and is identical to what @jacobadenbaum posted, i.e. it acts as a continue. The reason is that do defines an anonymous function and thus the return only returns from that anonymous function and does not exit the surrounding block somehow.

1 Like
5

I think the main point of @GunnarFarneback’s post was to explain where the ERROR: syntax: break or continue outside loop comes from: the continue is inside of the anonymous function, not (directly) in the (implicit) loop. This is orthogonal to the original question of actually emulating continue or break in something like pmap.

A clearer example for this error would be to contrast

julia> for x in 1:10
           x < 3 || break
           println(x)
       end
1
2

with

julia> foreach(1:10) do x
           x < 3 || break
           println(x)
       end
ERROR: syntax: break or continue outside loop

i.e.

julia> function f(x)
           x < 3 || break
           println(x)
       end
       foreach(f, 1:10)

where it’s obvious that the break in f makes no sense.

1 Like
6

It would help if you clarify exactly what you’d like to happen, but assuming its that for some inputs you just fail/skip the computation, the pattern I usually use for that kind of thing is:

status = collect(skipmissing(pmap(as) do a
    if a < 1
        return missing
    else
        return a^2 # or whatever youre doing
    end
end))