SharedArray undefined on worker

General Usage
1

I have what I thought was a really basic usage of SharedArrays. But it says the array is undefined on the worker processes and I have no idea why it would be. I have the following code in a file “test.jl”. I run it with julia -p 4 test.jl

using SharedArrays

const arr = SharedArray{Int64, 2}((5,5))

for i in 1:5
    arr[i, :] = fill(i, 5)
end

@everywhere function test()
    return arr[myid(), :]
end 

remotecall_fetch(test, 2)

it exits with

ERROR: LoadError: On worker 2:
UndefVarError: arr not defined
2

You need to define and load “arr” and SharedArrays on all processes.

@everywhere using SharedArrays
@everywhere arr = SharedArray{Int64, 2}((5,5))
3

No, that creates 5 different arr instead of one array that is shared. You can run this if you don’t believe me

@everywhere using SharedArrays
@everywhere arr = SharedArray{Int64, 2}((5,5))

@everywhere function test()
    arr[myid(), :] = fill(myid(), 5)
end 
println(arr)
test()
println(arr)
remotecall_fetch(test, 3)
println(arr)

If it were shared, the last print of arr should be [1 1 1 1 1; 0 0 0 0 0; 3 3 3 3 3; 0 0 0 0 0; 0 0 0 0 0] but its not

4

My mistake. Thanks for pointing that out.

Add @distributed to your loop

So using your original code

arr = SharedArray{Int64, 2}((5,5))

@distributed for i in 1:5
    arr[i, :] = fill(i, 5)
end

@everywhere function test()
    return arr[myid(), :]
end 
remotecall_fetch(test, 2)

Should return what you expect

5

I don’t think that’s right. I think the @distributed macro just creates copies similar to how @spawnat does. You can take a look at the documentation here Parallel Computing · The Julia Language. It talks about implicit data movement.

If for example, you add arr[2,2] = 10 after the distributed loop but before the test function, the remote call and fetch still only outputs [2, 2, 2, 2, 2]. Additionally, if you run your code with a regular array instead of a shared array (see below), it works the same which is evidence for the implicit data movement.

arr = Array{Int64, 2}(undef, 5,5)

@distributed for i in 1:5
    arr[i, :] = fill(i, 5)
end
arr[2,2] = 10
@everywhere function test()
    return arr[myid(), :]
end 
println(remotecall_fetch(test, 2))
6

My understanding is that you avoid implicit data movement if you loop through a SharedArray in this way. But I may have to spend a little bit more time reading the documentation.

With that said, This should be a posted as a new question; my differ based on whether I run the code as a script or via the REPL.

Via REPL (using Julia V1.3.0) I get [2,10,2,2,2] when I run the following

julia> using Distributed,SharedArrays

julia> addprocs(3)
3-element Array{Int64,1}:
 2
 3
 4

julia> @everywhere function test()
           return arr[myid(), :]
       end

julia> arr = SharedArray{Int64}(5, 5)
5×5 SharedArray{Int64,2}:
 0  0  0  0  0
 0  0  0  0  0
 0  0  0  0  0
 0  0  0  0  0
 0  0  0  0  0

julia> @distributed for i = 1:5
           
           arr[i, :] = fill(i, 5)
       end
Task (runnable) @0x00007f0856f57d00

julia> arr[2,2] = 10
10

julia> println(remotecall_fetch(test, 2))
[2, 10, 2, 2, 2]

however, running this as a script returns [2,2,2,2,2].

7

That’s very weird. Going back to my original example, it appears to work if you add a fetchfrom the process of interest first. I might submit an issue on github since it doesn’t make sense.

@everywhere using SharedArrays

const arr = SharedArray{Int64, 2}((5,5))

for i in 1:5
    arr[i, :] = fill(i, 5)
end

@everywhere function test()
    arr[myid(), myid()] = 10
    return arr[myid(), :]
end 

println(remotecall_fetch(test, 2)) # doesn't work
@fetchfrom 2 arr
println(remotecall_fetch(test, 2)) # works now
println(remotecall_fetch(test, 3)) # still doesn't work on this worker though
println(arr)
9

Hi!

There’s also the possibility to just broadcast the declaration, e.g.,

for w in workers()
	@spawnat(w,arr)
end
1 Like