Can't write to distributed array

General Usage
1

I’ve been experimenting with distributed arrays. I can create them just fine, but when I try to have a worker write to a part of a distributed array that it owns, I get an setindex! not defined for DArray error.

Here’s a MWE, running on 2 worker processes:

julia> testD = @DArray [i+j for i = 1:5, j = 1:5]
5×5 DArray{Int64,2,Array{Int64,2}}:
 2  3  4  5   6
 3  4  5  6   7
 4  5  6  7   8
 5  6  7  8   9
 6  7  8  9  10

julia> [@fetchfrom p localindices(testD) for p in workers()]
2-element Array{Tuple{UnitRange{Int64},UnitRange{Int64}},1}:
 (1:5, 1:3)
 (1:5, 4:5)

julia> remotecall_wait(D->D[1,1] = 0,2,testD)
ERROR: On worker 2:
setindex! not defined for DArray{Int64,2,Array{Int64,2}}
Rest of error 
error at ./error.jl:42
error_if_canonical_setindex at ./abstractarray.jl:1083
setindex! at ./abstractarray.jl:1072 [inlined]
#161 at ./none:1
#108 at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.4/Distributed/src/process_messages.jl:309
run_work_thunk at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.4/Distributed/src/process_messages.jl:79
run_work_thunk at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.4/Distributed/src/process_messages.jl:88
#94 at ./task.jl:358
Stacktrace:
 [1] remotecall_wait(::Function, ::Distributed.Worker, ::DArray{Int64,2,Array{Int64,2}}; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.4/Distributed/src/remotecall.jl:432
 [2] remotecall_wait(::Function, ::Distributed.Worker, ::DArray{Int64,2,Array{Int64,2}}) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.4/Distributed/src/remotecall.jl:423
 [3] remotecall_wait(::Function, ::Int64, ::DArray{Int64,2,Array{Int64,2}}; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.4/Distributed/src/remotecall.jl:444
 [4] remotecall_wait(::Function, ::Int64, ::DArray{Int64,2,Array{Int64,2}}) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.4/Distributed/src/remotecall.jl:444
 [5] top-level scope at none:0

I thought I was able to have process 2 write to the [1,1] element of the array, since it’s in the part of the array that it owns.

How do I enable a process to write to its part of a DArray?

2

Use the localpart of the DArray while assigning, this makes it explicit that a process only assigns to the section allocated to it.

julia> remotecall_wait(D->localpart(D)[1,1] = 0,2,testD)
Future(2, 1, 117, nothing)

julia> testD
5×5 DArray{Int64,2,Array{Int64,2}}:
 0  3  4  5   6
 3  4  5  6   7
 4  5  6  7   8
 5  6  7  8   9
 6  7  8  9  10
1 Like
3

Got it, thanks. I didn’t realize this had to be made explicit