# Copying variable to all remote processes?

**URL:** <https://discourse.julialang.org/t/copying-variable-to-all-remote-processes/26518>\
**Category:** General Usage\
**Tags:** parallel\
**Created:** [July 18, 2019, 8:35pm UTC](https://discourse.julialang.org/t/copying-variable-to-all-remote-processes/26518 "2019-07-18T20:35:36Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![vlandau](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vlandau/32/8986_2.png) [@vlandau](https://discourse.julialang.org/u/vlandau)\
**Post date:** [July 18, 2019, 8:35pm UTC](https://discourse.julialang.org/t/copying-variable-to-all-remote-processes/26518/1 "2019-07-18T20:35:36Z")

</div>

Hello all,

I’m working on some code where each remote worker needs to compute an array by accumulating values, and the arrays then need to be summed. Below is an absolute minimum working example to demonstrate the problem I’m encountering.

Obviously, the code below (lines 5 and 6) will not work because I did not use `@everywhere` when defining `q`. In my program though, “`q`” is the result of many functions and other variables, so I’m hoping there is another way to copy `q` to all workers without needing to use `@everywhere` in front of every one of `q`’s dependencies.

Any guidance or suggestions would be much appreciated. Thanks!

```julia
using Distributed
addprocs(2)

q = reshape(rand(9), 3, 3)
@everywhere nrow = eval(size(q, 1))
@everywhere ncol = eval(size(q, 2))
for i in workers()
    @spawnat i eval(:(a = fill(0, nrow, ncol)))
end

```

---

<div class="post-metadata">

**Author:** ![vlandau](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vlandau/32/8986_2.png) [@vlandau](https://discourse.julialang.org/u/vlandau)\
**Post date:** [July 18, 2019, 8:50pm UTC](https://discourse.julialang.org/t/copying-variable-to-all-remote-processes/26518/2 "2019-07-18T20:50:26Z")

</div>

In fact, I believe using `@everywhere` on all preceding variables will not work in my program because workers are created after some variables have already been defined.

---

<div class="post-metadata">

**Author:** ![vlandau](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vlandau/32/8986_2.png) [@vlandau](https://discourse.julialang.org/u/vlandau)\
**Post date:** [July 18, 2019, 9:21pm UTC](https://discourse.julialang.org/t/copying-variable-to-all-remote-processes/26518/3 "2019-07-18T21:21:08Z")

</div>

After some random trial and error, this appears to work, but I’m not exactly sure why. 🙃

```julia
using Distributed
addprocs(2)
q = fill(rand(9), 3, 3)

@everywhere nrows = @eval size($q, 1)
@everywhere ncols = @eval size($q, 2)
for i in workers()
    @spawnat i eval(:(a = fill(0, nrows, ncols)))
end

```

---

<div class="post-metadata">

**Author:** ![greg\_plowman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/greg_plowman/32/8100_2.png) [@greg\_plowman](https://discourse.julialang.org/u/greg_plowman)\
**Post date:** [July 18, 2019, 9:31pm UTC](https://discourse.julialang.org/t/copying-variable-to-all-remote-processes/26518/4 "2019-07-18T21:31:15Z")

</div>

You can avoid the `@eval` and just interpolate the sizes into @everywhere:

```julia
@everywhere nrows = $(size(q, 1))
@everywhere ncols = $(size(q, 2))

```

---

<div class="post-metadata">

**Author:** ![vlandau](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vlandau/32/8986_2.png) [@vlandau](https://discourse.julialang.org/u/vlandau)\
**Post date:** [July 19, 2019, 3:34pm UTC](https://discourse.julialang.org/t/copying-variable-to-all-remote-processes/26518/5 "2019-07-19T15:34:54Z")

</div>

The solutions posted do work in the MWE, but for some reason they don’t work in my Julia package. Could it be something to do with function dependencies? Here’s the code that is encountering the problem. This is all inside a function, by the way, and `parse_cfg` and `read_ascii` are both functions in my package. FWIW, all of the code is available on GitHub [here](https://github.com/Circuitscape/Omniscape.jl/tree/parallel).

```julia
cfg = parse_cfg(path)
calc_flow_potential = cfg["calc_flow_potential"] == "true"
parallelize = cfg["parallelize"] == "true"
n_workers = parse(Int64, cfg["max_parallel"])
sources_raw = float(read_ascii("$(cfg["source_file"])"))

if parallelize
    println("Starting up Omniscape to use $(n_workers) processes in parallel")
    myaddprocs(n_workers)

    @everywhere nrows_remote = $(size(sources_raw, 1))
    @everywhere ncols_remote = $(size(sources_raw, 2))

    for i in workers()
        @spawnat i eval(:(cum_currmap = fill(0.,
                                             nrows_remote,
                                             ncols_remote)))
    end

    if calc_flow_potential
        for i in workers()
            @spawnat i eval(:(fp_cum_currmap = fill(0.,
                                                    nrows_remote,
                                                    ncols_remote)))
        end
    end
end

```

`myaddprocs(n)` is simply `addprocs(n)` then `@everywhere Core.eval(Main, :(import Omniscape))`

---

<div class="post-metadata">

**Author:** ![vlandau](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vlandau/32/8986_2.png) [@vlandau](https://discourse.julialang.org/u/vlandau)\
**Post date:** [July 19, 2019, 4:18pm UTC](https://discourse.julialang.org/t/copying-variable-to-all-remote-processes/26518/6 "2019-07-19T16:18:16Z")

</div>

# EDIT

I did get everything sorted by using `remotefetch()` instead, FWIW!

* * *

I found a _super_ hacky workaround using `pmap` to effectively copy the variables to remote workers using the following: (this is clearly not an ideal solution)

```julia
function copyvars(i, array)
    global nrows_remote = size(array, 1)
    global ncols_remote = size(array, 2)
end

pmap(x -> copyvars(x, sources_raw), 1:50)

```

Any reason why this works while the script in the post just above doesn’t? Thanks again for any insight!