Invoking "remote" variables in remote computations

General Usage
1

Let’s suppose in a distributed computing task, for each worker or process we’d like to compute some data (for example in the form of a matrix A) which might or might not be unique for that worker and will be useful later for further computations. A is relatively inexpensive to compute but expensive to send across processes, in that sense we want to store the data as a local variable in the remote worker.

To my understanding @spawnat and remotecall use the computing power of the workers, but without affecting their workspace. In @spawnat workers()[1] A = rand(2,2), A is not stored in workers()[1]. To do so, the @everywhere macro is a candidate for the job.

@everywhere workers() A = rand(2,2)

Then problem I find then is how to invoke A for computations (say A*b with a local vector b), as @spawnat and remotecall read only local variables.

Using @everywhere let’s us invoke A into the computation

b = rand(2)
@everywhere workers() x = A*$b

But how should then the x’s be fetched back? I thought through a RemoteChannel, but I couldn’t set one up.

2

One way is to pass in a RemoteChannel reference to the workers, eg:

julia> r = RemoteChannel(()->Channel{Any}(nworkers()));

julia> @everywhere workers() function Atimesb(b,r)
           A = rand(2,2)
           x = A*b;
           put!(r,b)
       end

julia> @everywhere workers() Atimesb($b,$r)

# collect the results on the master
julia> result = [take!(r) for i=1:nworkers()]
2-element Array{Array{Float64,1},1}:
 [0.16141836526706888, 0.12093141352370562]
 [0.16141836526706888, 0.12093141352370562]