Correct usage of SharedArray and @parallel?

General Usage
1

In defining a function, I created a SharedArray and use @parallel to fill each of its elements. At last, I want to return the mean of these elements by calling mean on the SharedArray. However, it doesn’t seem to work as expected. Here’s a MWE:

function foo(m::Int64)
    z = SharedArray(Float64, m)
    @parallel for i = 1:m
        z[i] = 1.0
    end
    mean(z)
end

On my Mac, calling foo(10) for the first time gives result 0.0, whereas 0.7 afterwards.

If the function returns the SharedArray instead, the output is correct:

function bar(m::Int64)
    z = SharedArray(Float64, m)
    @parallel for i = 1:m
        z[i] = 1.0
    end
   z
end

Calling bar(10) gives the expected result.

To add even more confusion, z = bar(10); mean(z) and mean(bar(10)) give different results. The former gives what is expected, whereas the latter behaves as foo(10), i.e. returning 0.0 for the first call and 0.7 afterwords.

I’m totally confused.

2

Try @sync.
If you don’t have a reduction operator in @parallel for, it doesn’t wait for processes to complete.

function foo(m::Int64)
    z = SharedArray(Float64, m)
    @sync @parallel for i = 1:m
        z[i] = 1.0
    end
    mean(z)
end

http://docs.julialang.org/en/stable/manual/parallel-computing/#parallel-computing

As you could see, the reduction operator can be omitted if it is not needed. In that case, the loop executes
asynchronously, i.e. it spawns independent tasks on all available workers and returns an array of Future immediately
without waiting for completion. The caller can wait for the Future completions at a later point by calling fetch() on them, or
wait for completion at the end of the loop by prefixing it with @sync, like @sync @parallel for.
4 Likes
3

Thanks! It solves my problem. I think it may be of great help if the document can give an example to warn new users.

4

You are the best expert to fix the documentation.

3 Likes