How to do `@parallel for i = 1:n, j = 1:m` like thing?

General Usage
1

I want to use multidimensional SharedArray to calculate in parallel.
From the documents (https://docs.julialang.org/en/stable/manual/parallel-computing/), the following code calculates in parallel for 1-dimensional array:

a = SharedArray{Float64}(10)
@parallel for i = 1:10
    a[i] = i
end

How can I do this for 2-dimensional array like the following?

# this is not runnable
a = SharedArray{Float64}(10, 5)
@parallel for i = 1:10, j = 1:5
    a[i, j] = i + j
end
2

The simplest method I know of is

a = SharedArray{Float64}(10, 5)
@parallel for k in eachindex(a)
    i, j = ind2sub(a, k)
    a[k] = i + j
    # or: a[i, j] = i + j
end

If you don’t need the subscripts themselves you can (obviously) simply do

a = SharedArray{Float64}(10, 5)
@parallel for k in eachindex(a)
    a[k] = k
end

In the upcoming 0.7 (didn’t work in 0.6 for me) you can use @parallel with CartesianIndices:

a = SharedArray{Float64}(10, 5)
r = CartesianIndices(size(a))
@parallel for k in r
    i, j = k.I
    a[k] = i + j
end
2 Likes
3

I didn’t know about ind2sub. Thanks for your help!

4

Depending on the dimensions of your shared array, you might consider nested for loops:

a = SharedArray{Float64}(10, 5)
@parallel for i = 1:10
    for j = 1:5
        a[i, j] = i + j
    end
end

This partitions the outer loop across workers, and doesn’t require ind2sub or equivalent.

5

Can this partition the inner loop? I want to partition for each (i, j) and I need to specify (i, j).

6

No. (But not really sure what you mean here)

The outer loop will be partitioned across the workers.
So in this example (10xN array), say you have 5 workers. Then each worker will process 2 iterations of the outer loop (i.e. each worker will execute the full inner loop twice)
In general this should work if (outer loop dimension) > (number of workers)

7 
# parallel calculation

@everywhere function test(i, j)
    sleep(0.01)
    println(i, j)
end

@everywhere function test1(i, m)
    @sync @parallel for j = 1:m
        test(i, j)
    end
end

function test_series(n, m)
    for i = 1:n
        for j = 1:m
            test(i, j)
        end
    end
end

function test_parallel_single_loop(n, m)
    @sync @parallel for i = 1:n
        for j = 1:m
            test(i, j)
        end
    end
end

function test_parallel_double_loop(n, m)
    @sync @parallel for i = 1:n
        test1(i, m)
    end
end

n = 10
m = 10

function calc()
    @time test_series(n, m)
    @time test_parallel_single_loop(n, m)
    @time test_parallel_double_loop(n, m)
    @time test_series(n, m)
    @time test_parallel_single_loop(n, m)
    @time test_parallel_double_loop(n, m)
end

calc()

I mean the difference among them.