# Using @everywhere include() from Distributed.jl?

**URL:** <https://discourse.julialang.org/t/using-everywhere-include-from-distributed-jl/87237>\
**Category:** General Usage\
**Tags:** distributed, weave\
**Created:** [September 13, 2022, 9:01pm UTC](https://discourse.julialang.org/t/using-everywhere-include-from-distributed-jl/87237 "2022-09-13T21:01:53Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [September 13, 2022, 9:01pm UTC](https://discourse.julialang.org/t/using-everywhere-include-from-distributed-jl/87237/1 "2022-09-13T21:01:53Z")

</div>

I’ve got a Weave document (.jmd) and I’m using

```julia
@everywhere include("./utilities.jl")

```

There’s an error that says the remote machine is trying to load a file not in the current directory, but the same directory as on my local machine, which doesn’t exist on the remote machine.

documentation for @everywhere macro says:

```julia
  Similar to calling remotecall_eval(Main, procs, expr), but with two extra
  features:

  - `using` and `import` statements run on the calling process first, to ensure
    packages are precompiled.
  - The current source file path used by `include` is propagated to other processes.

```

I think the second feature is a misfeature. If I wanted it to read from the same path as on my local computer, I’d send it an absolute path… instead it takes the relative path, converts it to an absolute path, and sends that to the remote machine, which fails since my home directory is not mounted on the remote machine…

What’s the workaround? and should this be a bug?

At the moment, I’m trying to work around with Distributed.remotecall\_eval(…) we’ll see how it goes.

---

<div class="post-metadata">

**Author:** ![suavesito](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/suavesito/32/34386_2.png) [@suavesito](https://discourse.julialang.org/u/suavesito)\
**Post date:** [September 13, 2022, 9:14pm UTC](https://discourse.julialang.org/t/using-everywhere-include-from-distributed-jl/87237/2 "2022-09-13T21:14:50Z")

</div>

From the same documentation, maybe you can try the next.

```julia
using Distributed
Distributed.remotecall_eval(Main, procs(), :(include("./utilities.jl")))

```

My guess is that in HPC environments, i.e. super computers, you would run `@everywhere` in the cluster as a script and probably all the machines would be configured similarly, including in terms of directories and filesystem structure.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [September 13, 2022, 9:17pm UTC](https://discourse.julialang.org/t/using-everywhere-include-from-distributed-jl/87237/3 "2022-09-13T21:17:44Z")

</div>

Yes, the remotecall\_eval trick _sort of_ worked. It did get the file included, but then things borked because it’s a Weave.jl document… I’ve got a separate post on that topic. Note that remotecall\_eval is not exported by Distributed and seems kind of less friendly.

I think the assumption that all machines are configured exactly the same is flawed. At the least it should be possible to override this behavior for include().

---

<div class="post-metadata">

**Author:** ![suavesito](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/suavesito/32/34386_2.png) [@suavesito](https://discourse.julialang.org/u/suavesito)\
**Post date:** [September 13, 2022, 9:25pm UTC](https://discourse.julialang.org/t/using-everywhere-include-from-distributed-jl/87237/4 "2022-09-13T21:25:13Z")

</div>

> [@dlakelan](#):
>
> At the least it should be possible to override this behavior for include().

You can fill an issue in the [JuliaLang repository](https://github.com/julialang/julia/issues), I think it would be a nice option.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [September 13, 2022, 9:28pm UTC](https://discourse.julialang.org/t/using-everywhere-include-from-distributed-jl/87237/5 "2022-09-13T21:28:51Z")

</div>

Ok, I did that

> <https://github.com/JuliaLang/julia/issues/46748>
>
> According to tests and the documentation string:
> 
> \`\`\`
> Similar to calling re…motecall\_eval(Main, procs, expr), but with two extra
> features:
> 
> - \`using\` and \`import\` statements run on the calling process first, to ensure
> packages are precompiled.
> - The current source file path used by \`include\` is propagated to other processes.
> \`\`\`
> This last "feature" overrides the path even if that isn't desired behavior. I discovered this when doing
> 
> \`\`\`
> @everywhere include("./utilities.jl")
> \`\`\`
> and I expected it to read from the "current directory" that the remote julia process started in, but in fact it tried to read from my home directory (same directory as on my machine where the whole process was initiated) however that path doesn't exist remotely and isn't the desired behavior. It would be nice to have an option to not have \`@everywhere include(...)\` override the path.