# Sending modules from a local julia file to remote machines

**URL:** https://discourse.julialang.org/t/sending-modules-from-a-local-julia-file-to-remote-machines/8088
**Category:** General Usage
**Created:** [January 1, 2018, 6:43am UTC](https://discourse.julialang.org/t/sending-modules-from-a-local-julia-file-to-remote-machines/8088 "2018-01-01T06:43:15Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Snir\_Gazit](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/snir_gazit/32/11027_2.png) [@Snir\_Gazit](https://discourse.julialang.org/u/Snir_Gazit)
#### Post date: [January 1, 2018, 6:43am UTC](https://discourse.julialang.org/t/sending-modules-from-a-local-julia-file-to-remote-machines/8088/1 "2018-01-01T06:43:15Z")

</div>

Hi,

I’m trying to run a Julia program comprising multiple modules on a remote machine. The relevant Julia files are available locally but not on the remote machine.  
Adding processes on the remote machines seems to work fine. But trying to include Julia code fails. Explicitly, with

```julia
@everywhere include("MyModule.jl")

```

Julia complains that it cannot find the above Julia module file. Presumably, this happens because the file is not available on the remote machine.  
How can I make the modules and functions contained in my local Julia files available on the remote machine?

Thanks!

---

<div class="post-metadata">

### Author: ![dfdx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dfdx/32/120_2.png) [@dfdx](https://discourse.julialang.org/u/dfdx)
#### Post date: [January 1, 2018, 12:10pm UTC](https://discourse.julialang.org/t/sending-modules-from-a-local-julia-file-to-remote-machines/8088/2 "2018-01-01T12:10:25Z")

</div>

You may try something like this:

```julia
# read file as text
text = readstring("MyModule.jl")
# parse it into an expression
ex = parse(text)
# evaluate this expression on all workers
@everywhere eval(ex)

```

The downside is that if MyModule refers to other files or modules, they won’t be sent. You can handle some of such cases by recursively analyzing the content of included files, but it’s not very robust.

Alternatively, you can make MyModule available for downloading on all workers and then simply run:

```julia
Pkg.add("MyModule")

```

or

```julia
Pkg.checkout("git://<public or private repo>/MyModule.jl.git")

```

This way Julia will handle all the dependencies automatically. One disadvantage of this approach is that packages are installed globally which may be inconvenient on shared workers.

---

<div class="post-metadata">

### Author: ![Snir\_Gazit](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/snir_gazit/32/11027_2.png) [@Snir\_Gazit](https://discourse.julialang.org/u/Snir_Gazit)
#### Post date: [January 1, 2018, 2:55pm UTC](https://discourse.julialang.org/t/sending-modules-from-a-local-julia-file-to-remote-machines/8088/3 "2018-01-01T14:55:14Z")

</div>

Thanks for the detailed explanations and solutions. For the second solution, suppose I started several workers on the remote machine, wouldn’t they all download the package simultaneously? If so, would writing in parallel to the remote HD be problematic?

---

<div class="post-metadata">

### Author: ![dfdx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dfdx/32/120_2.png) [@dfdx](https://discourse.julialang.org/u/dfdx)
#### Post date: [January 2, 2018, 12:22am UTC](https://discourse.julialang.org/t/sending-modules-from-a-local-julia-file-to-remote-machines/8088/4 "2018-01-02T00:22:19Z")

</div>

I don’t know if the built-in package manager can handle simultaneous installation / lock on packages, but I would simply run `Pkg.add()` on each worker synchronously, something like this:

```julia
for w in workers()
    remotecall_wait(Pkg.add, w, "MyModule")
end

```

Since you need to install packages only once, this will be slow the first time you do it, but then `Pkg` will only check the package for existence.

Alternatively, you can remove duplicated from the list of worker hosts and run `Pkg.add()` only on a list of unique ones.

---

<div class="post-metadata">

### Author: ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)
#### Post date: [September 20, 2018, 2:31am UTC](https://discourse.julialang.org/t/sending-modules-from-a-local-julia-file-to-remote-machines/8088/5 "2018-09-20T02:31:55Z")

</div>

```julia
# Include a file on a path not available on a remote worker
include_remote(path, 2)

```

From [https://github.com/ChrisRackauckas/ParallelDataTransfer.jl](https://github.com/ChrisRackauckas/ParallelDataTransfer.jl)
