The problem I have is how to define functions that modify parts of SharedArrays. What I want to achieve is the following:
using Distributed
addprocs(2)
@everywhere using SharedArrays
@everywhere function temp(trace)
Nt = size(trace, 1)
Fs = Vector{Any}(undef, Nt)
for n = 1:Nt
i = 1
Fs[n] = res -> begin
trace[n, i] = res
i += 1
end
end
return Fs
end
trace = SharedArray{Int}(3,4)
Fs = temp(trace)
for i = 1:4
for n = 1:3
# Large computation and save result through a callback
res = n+i
Fs[n](res)
end
end
show(stdout, "text/plain", trace)
println("\n")
which yields
3×4 SharedArray{Int64,2}:
2 3 4 5
3 4 5 6
4 5 6 7
However, when trying the distributed version:
@everywhere trace = SharedArray{Int}(3,4)
@everywhere Fs = temp(trace)
for i = 1:4
@distributed for n = 1:3
Fs[n](n + i)
end
end
show(stdout, "text/plain", trace)
println("\n")
I get
3×4 SharedArray{Int64,2}:
0 0 0 0
0 0 0 0
0 0 0 0
Seems like Fs
is not defined on the different processes. I think this could be solved using RemoteChannels
but how would I do this using SharedArrays
?
edit:
Tried with pmap
as well
trace = SharedArray{Int}(3,4)
Fs = temp(trace)
for i = 1:4
pmap(enumerate(Fs)) do (n, F)
F(n + i)
end
end
show(stdout, "text/plain", trace)
println("\n")
With the result
3×4 SharedArray{Int64,2}:
5 0 0 0
6 0 0 0
7 0 0 0
Meaning that i
does not increment in the loop where Fs
is defined.