# Loading modules on remote workers in Julia 1.0/.7

**URL:** <https://discourse.julialang.org/t/loading-modules-on-remote-workers-in-julia-1-0-7/13312>\
**Category:** General Usage\
**Created:** [August 12, 2018, 5:15pm UTC](https://discourse.julialang.org/t/loading-modules-on-remote-workers-in-julia-1-0-7/13312 "2018-08-12T17:15:53Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [August 12, 2018, 5:15pm UTC](https://discourse.julialang.org/t/loading-modules-on-remote-workers-in-julia-1-0-7/13312/1 "2018-08-12T17:15:53Z")

</div>

I’m having difficulty loading a module on remote workers. Suppose I have a file called myModule.jl in a directory assigned to variable called path. In the past, I simply added path to `LOAD_PATH` and entered `@everywhere using myModule`. This does not work anymore.

#myModule.jl

```julia
module myModule
    export myFunction
    myFunction(x) = x + 1
end

```

#top level code in run.jl

```julia
path = "/home/dfish/myFolder"
push!(LOAD_PATH,path)
using Distributed
addprocs(4)
@everywhere using myModule
pmap(myFunction,1:1000)

```

When I run the code, I receive the following error:

```julia
ERROR: LoadError: On worker 2:
ArgumentError: Package myModule not found in current path:
- Run `Pkg.add("myModule")` to install the myModule package.

require at ./loading.jl:817
eval at ./boot.jl:319 
...

```

The error message indicates that I need to add it as a package. However, I can load it into the primary processor without issue:

```julia
 using myModule

 myFunction(2)
  3

```

As you can see, my LOAD\_PATH and file system are both properly configured:

```julia
 readdir(path)
  2-element Array{String,1}:
 "myModule.jl"
 "run.jl"

 LOAD_PATH
  5-element Array{String,1}:
 "@"
 "@v#.#"
 "@stdlib"
 "/home/dfish/Projects/ACT-R-Likelihood/FFT Models/FFT"
 "/home/dfish/myFolder"

```

I tried `@everywhere include("myModule.jl")` but the code failed at `pmap.`

---

<div class="post-metadata">

**Author:** ![greg\_plowman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/greg_plowman/32/8100_2.png) [@greg\_plowman](https://discourse.julialang.org/u/greg_plowman)\
**Post date:** [August 13, 2018, 2:52am UTC](https://discourse.julialang.org/t/loading-modules-on-remote-workers-in-julia-1-0-7/13312/2 "2018-08-13T02:52:33Z")

</div>

Code loading was revamped during 0.7 development and I think loading code from node 1 is no longer supported.

My understanding is that each worker must have access to a (possibly shared) filesystem that contains the module, and loads code locally.

Perhaps the “old” behavior of loading from node 1 is a feature left to be implemented in package.

See some discussion here:  
[https://github.com/JuliaLang/julia/pull/19738](https://github.com/JuliaLang/julia/pull/19738)  
[https://github.com/JuliaLang/julia/pull/19910](https://github.com/JuliaLang/julia/pull/19910)  
[https://github.com/JuliaLang/julia/pull/20120](https://github.com/JuliaLang/julia/pull/20120)

---

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [August 13, 2018, 8:52am UTC](https://discourse.julialang.org/t/loading-modules-on-remote-workers-in-julia-1-0-7/13312/3 "2018-08-13T08:52:56Z")

</div>

Thanks for your reply. Unfortunately, the discussion did not clarify how to load my own modules to other workers. Evidently in Julia 1/.7, `using` can load any module to the first node and can load a module that is a package to any node, as this shows:

```julia
using Distributed
addprocs(4)
@everywhere using Distributions
@everywhere f(x) = rand(Normal(0,1),1000)
pmap(f,1:10)

```

I find this behavior to be confusing in contrast to previous versions in which `using` loaded modules in all situations. Do you know how I can make my own `loadEverywhere()` function to replace this lost functionality?

---

<div class="post-metadata">

**Author:** ![greg\_plowman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/greg_plowman/32/8100_2.png) [@greg\_plowman](https://discourse.julialang.org/u/greg_plowman)\
**Post date:** [August 13, 2018, 9:52am UTC](https://discourse.julialang.org/t/loading-modules-on-remote-workers-in-julia-1-0-7/13312/4 "2018-08-13T09:52:13Z")

</div>

I think your example of `using Distributions` works for _local_ workers because all the workers have access to the local package directory.

I suspect this would not work for _remote_ workers if `Distributions` is not installed on those remote workers.

A workaround for loading your own modules on _local_ workers might be something like:

```julia
using Distributed
addprocs()
module_dir = "/home/dfish/myFolder"
@everywhere push!(LOAD_PATH, $module_dir)
@everywhere using myModule
pmap(myFunction, 1:1000)

```

AFAICT, the easiest way to load code on remote workers is to provide shared access to the local machine’s packages and modules.

---

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [August 13, 2018, 10:00am UTC](https://discourse.julialang.org/t/loading-modules-on-remote-workers-in-julia-1-0-7/13312/5 "2018-08-13T10:00:33Z")

</div>

Thanks. Adding `@everywhere push!(LOAD_PATH, $module_dir)` fixed the issue. I tried `@everywhere push!(LOAD_PATH,path)` at one point, but it looks like the `$` is critical to its operation. Would you be willing to explain how I provide shared access? This is new to me.

---

<div class="post-metadata">

**Author:** ![floswald](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/floswald/32/195_2.png) [@floswald](https://discourse.julialang.org/u/floswald)\
**Post date:** [August 13, 2018, 10:11am UTC](https://discourse.julialang.org/t/loading-modules-on-remote-workers-in-julia-1-0-7/13312/6 "2018-08-13T10:11:21Z")

</div>

from `?addprocs` you see that calling `addprocs(4)`

> addprocs(4) will add 4 processes on the local machine.

so the issue here is not a shared file system. all processes on your machine have access to the same, unique, file system, i.e. your disk. the only thing that changed to previous julia versions is that you need to push stuff onto the LOAD\_PATH on all workers, as indeed @greg_plowman said.

If you had 2 machines, say, you would have to worry about this. either, you maintain exact equal code bases on both machines (via git probably), or you would have a file share between them, such that `"/home/dfish/myFolder"` is the exact same file on both of them, for example. on ubuntu this is not too bad to do yourself: [SettingUpNFSHowTo - Community Help Wiki](https://help.ubuntu.com/community/SettingUpNFSHowTo).

---

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [August 13, 2018, 10:14am UTC](https://discourse.julialang.org/t/loading-modules-on-remote-workers-in-julia-1-0-7/13312/7 "2018-08-13T10:14:07Z")

</div>

Excellent. Thanks for your explanation. I will make Greg’s post as the answer.

---

<div class="post-metadata">

**Author:** ![paalon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paalon/32/5784_2.png) [@paalon](https://discourse.julialang.org/u/paalon)\
**Post date:** [July 25, 2019, 8:51am UTC](https://discourse.julialang.org/t/loading-modules-on-remote-workers-in-julia-1-0-7/13312/8 "2019-07-25T08:51:15Z")

</div>

I also forgot to add `@everywhere` to `push!(LOAD_PATH, pwd())`. Thanks.