Parallel code seems slow

General Usage Performance
1

Hello,

I have a code that I want to run multiple times with different random inputs to compute an average overall all iterations.

To get more familiar with parallel programming in Julia, I tried to play around with a small example :

n = 10^3

A = SharedArray{Int}(n)

@time begin
    @sync @parallel for i in 1:n
        A[i] = i
    end
end

I ran it a few times and I got some disappointing results.

For n = 10^3, I got

39
39984×224 40.8 KB

For n = 10^5, I got

25
25994×240 41.2 KB

For n = 10^7, I got

16
161002×248 43.5 KB

It seems a bit faster to use julia -p 2 than julia -p 1, but it is definitely faster to use only julia.

I also try without the parallel stuff and obtain faster times. More precisely, for n = 10^3, 10^5, 10^7, I get

47
471004×282 45.4 KB

Finally, I also ran the code on a cluster and got similar results.

Am I doing something wrong or is there a problem with parallel programming in Julia?

2

If you’re only running the code once, then you’re probably mostly just timing compilation. At the very least, I would suggest putting the code in a function and then using GitHub - JuliaCI/BenchmarkTools.jl: A benchmarking framework for the Julia language to get a reliable time estimate.

Also, I’m far from an expert in parallel computing, but in an example like the one you’ve posted, in which every operation is trivially cheap, I would suspect that the overhead of sending data between processes vastly outweighs the performance gain from computing i = i at each iteration.

1 Like
3

You will need to have your code inside a function and compile it first:

n = 10^7

A = SharedArray{Int}(n)

function myfill!(A,n)
    @sync @parallel for i in 1:n
        A[i] = i
    end
end
#compile
myfill!(A,20)
#run 
@time myfill!(A,n)
println(A[end])

Try timing with the code above.

4

Yeah, that was exactly the problem. I taught that the @parallel and @sync would be pre-compile. I was wrong.

Thanks !