# @everywhere using MyLib fails when procs on remote node

**URL:** <https://discourse.julialang.org/t/everywhere-using-mylib-fails-when-procs-on-remote-node/4515>\
**Category:** General Usage\
**Tags:** package, parallel, cluster\
**Created:** [June 28, 2017, 1:03pm UTC](https://discourse.julialang.org/t/everywhere-using-mylib-fails-when-procs-on-remote-node/4515 "2017-06-28T13:03:46Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![alha02](https://avatars.discourse-cdn.com/v4/letter/a/6a8cbe/32.png) [@alha02](https://discourse.julialang.org/u/alha02)\
**Post date:** [June 28, 2017, 1:03pm UTC](https://discourse.julialang.org/t/everywhere-using-mylib-fails-when-procs-on-remote-node/4515/1 "2017-06-28T13:03:46Z")

</div>

I am having troubles loading my package on remote nodes with multiple procs:

```julia
julia> @everywhere using MyLib
INFO: Precompiling module MyLib.
WARNING: Node state is inconsistent: node 2 failed to load cache from /home/alha02/.julia/lib/v0.6/MyLib.ji. Got:
WARNING: can only precompile from node 1
WARNING: Node state is inconsistent: node 10 failed to load cache from /home/alha02/.julia/lib/v0.6/MyLib.ji. Got:
WARNING: can only precompile from node 1
WARNING: Node state is inconsistent: node 12 failed to load cache from /home/alha02/.julia/lib/v0.6/MyLib.ji. Got:
WARNING: can only precompile from node 1
WARNING: Node state is inconsistent: node 3 failed to load cache from /home/alha02/.julia/lib/v0.6/MyLib.ji. Got:
WARNING: can only precompile from node 1
WARNING: Node state is inconsistent: node 7 failed to load cache from /home/alha02/.julia/lib/v0.6/MyLib.ji. Got:
WARNING: can only precompile from node 1
WARNING: Node state is inconsistent: node 14 failed to load cache from /home/alha02/.julia/lib/v0.6/MyLib.ji. Got:
WARNING: can only precompile from node 1
WARNING: Node state is inconsistent: node 17 failed to load cache from /home/alha02/.julia/lib/v0.6/MyLib.ji. Got:
WARNING: can only precompile from node 1
WARNING: Node state is inconsistent: node 4 failed to load cache from /home/alha02/.julia/lib/v0.6/MyLib.ji. Got:
WARNING: can only precompile from node 1
WARNING: Node state is inconsistent: node 16 failed to load cache from /home/alha02/.julia/lib/v0.6/MyLib.ji. Got:
WARNING: can only precompile from node 1
WARNING: Node state is inconsistent: node 15 failed to load cache from /home/alha02/.julia/lib/v0.6/MyLib.ji. Got:
WARNING: can only precompile from node 1
WARNING: Node state is inconsistent: node 11 failed to load cache from /home/alha02/.julia/lib/v0.6/MyLib.ji. Got:
WARNING: can only precompile from node 1
WARNING: Node state is inconsistent: node 9 failed to load cache from /home/alha02/.julia/lib/v0.6/MyLib.ji. Got:
WARNING: can only precompile from node 1
WARNING: Node state is inconsistent: node 8 failed to load cache from /home/alha02/.julia/lib/v0.6/MyLib.ji. Got:
WARNING: can only precompile from node 1
WARNING: Node state is inconsistent: node 6 failed to load cache from /home/alha02/.julia/lib/v0.6/MyLib.ji. Got:
WARNING: can only precompile from node 1
WARNING: Node state is inconsistent: node 5 failed to load cache from /home/alha02/.julia/lib/v0.6/MyLib.ji. Got:
WARNING: can only precompile from node 1
WARNING: Node state is inconsistent: node 13 failed to load cache from /home/alha02/.julia/lib/v0.6/MyLib.ji. Got:
WARNING: can only precompile from node 1
ERROR: On worker 2:
can only precompile from node 1
compilecache at ./loading.jl:670
_require at ./loading.jl:456
require at ./loading.jl:398
_require_from_serialized at ./loading.jl:203
_require_search_from_serialized at ./loading.jl:236
_require at ./loading.jl:434
require at ./loading.jl:398
_require_from_serialized at ./loading.jl:203
_require_search_from_serialized at ./loading.jl:236
_require at ./loading.jl:434
require at ./loading.jl:398
eval at ./boot.jl:235
eval_ew_expr at ./distributed/macros.jl:116
#106 at ./distributed/process_messages.jl:268 [inlined]
run_work_thunk at ./distributed/process_messages.jl:56
macro expansion at ./distributed/process_messages.jl:268 [inlined]
#105 at ./event.jl:73
#remotecall_fetch#141(::Array{Any,1}, ::Function, ::Function, ::Base.Distributed.Worker, ::Expr, ::Vararg{Expr,N} where N) at ./distributed/remotecall.jl:354
remotecall_fetch(::Function, ::Base.Distributed.Worker, ::Expr, ::Vararg{Expr,N} where N) at ./distributed/remotecall.jl:346
#remotecall_fetch#144(::Array{Any,1}, ::Function, ::Function, ::Int64, ::Expr, ::Vararg{Expr,N} where N) at ./distributed/remotecall.jl:367
remotecall_fetch(::Function, ::Int64, ::Expr, ::Vararg{Expr,N} where N) at ./distributed/remotecall.jl:367
(::##1#3)() at ./distributed/macros.jl:102

...and 15 more exception(s).

Stacktrace:
 [1] sync_end() at ./task.jl:287
 [2] macro expansion at ./distributed/macros.jl:112 [inlined]
 [3] anonymous at ./<missing>:?

```

I have the following output from `versioninfo()` on the calling node (remote node is identical):

```julia
Julia Version 0.6.0
Commit 9036443 (2017-06-19 13:05 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU E5-2673 v3 @ 2.40GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, haswell)

```

Does anyone know what the problem might be? I’m not sure if this is a bug or if I missed something, so posting here before filing a bug report.

---

<div class="post-metadata">

**Author:** ![adamslc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adamslc/32/3452_2.png) [@adamslc](https://discourse.julialang.org/u/adamslc)\
**Post date:** [June 28, 2017, 6:22pm UTC](https://discourse.julialang.org/t/everywhere-using-mylib-fails-when-procs-on-remote-node/4515/2 "2017-06-28T18:22:15Z")

</div>

Have you installed `MyLib` on the remote node (and made sure it is up to date with your local copy)?

---

<div class="post-metadata">

**Author:** ![alha02](https://avatars.discourse-cdn.com/v4/letter/a/6a8cbe/32.png) [@alha02](https://discourse.julialang.org/u/alha02)\
**Post date:** [June 28, 2017, 6:55pm UTC](https://discourse.julialang.org/t/everywhere-using-mylib-fails-when-procs-on-remote-node/4515/3 "2017-06-28T18:55:01Z")

</div>

Yes, this was from a fresh install of `MyLib` on both nodes.

Edit: `using MyLib` works perfectly on both remote and local as well.

---

<div class="post-metadata">

**Author:** ![Michael\_Eastwood](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/michael_eastwood/32/669_2.png) [@Michael\_Eastwood](https://discourse.julialang.org/u/Michael_Eastwood)\
**Post date:** [June 28, 2017, 9:02pm UTC](https://discourse.julialang.org/t/everywhere-using-mylib-fails-when-procs-on-remote-node/4515/4 "2017-06-28T21:02:34Z")

</div>

I don’t think you need to use `@everywhere` to make the package available on all the nodes. For example:

```julia
$ julia -p 1
               _
   _ _ _(_)_ | A fresh approach to technical computing
  (_) | (_) (_) | Documentation: https://docs.julialang.org
   _ _ _| |_ __ _ | Type "?help" for help.
  | | | | | | |/ _` | |
  | | |_| | | | (_| | | Version 0.6.0 (2017-06-19 13:05 UTC)
 _/ |\ __'_|_|_|\__'_| |  
|__/ | x86_64-unknown-linux-gnu

julia> using Colors

julia> remotecall_fetch(colormatch, 2, 500)
XYZ{Float64}(0.0049,0.323,0.272)

```

---

<div class="post-metadata">

**Author:** ![alha02](https://avatars.discourse-cdn.com/v4/letter/a/6a8cbe/32.png) [@alha02](https://discourse.julialang.org/u/alha02)\
**Post date:** [June 29, 2017, 4:12am UTC](https://discourse.julialang.org/t/everywhere-using-mylib-fails-when-procs-on-remote-node/4515/5 "2017-06-29T04:12:26Z")

</div>

_facepalm_ Of course. I got rid of the error now, the warnings remain. But I suspect that they might be because of some connectivity issue. Thanks!

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [June 29, 2017, 3:27pm UTC](https://discourse.julialang.org/t/everywhere-using-mylib-fails-when-procs-on-remote-node/4515/6 "2017-06-29T15:27:01Z")

</div>

See also [https://github.com/JuliaLang/julia/pull/21718](https://github.com/JuliaLang/julia/pull/21718), which I thought should have eliminated this gotcha?

---

<div class="post-metadata">

**Author:** ![Michael\_Eastwood](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/michael_eastwood/32/669_2.png) [@Michael\_Eastwood](https://discourse.julialang.org/u/Michael_Eastwood)\
**Post date:** [June 29, 2017, 4:28pm UTC](https://discourse.julialang.org/t/everywhere-using-mylib-fails-when-procs-on-remote-node/4515/7 "2017-06-29T16:28:35Z")

</div>

I think on the first time you load a package it still needs to be just `using MyLib` otherwise all the nodes are trying to precompile the package and then write the .ji to the same location (if you have a shared home directory). Afterwards `@everywhere using MyLib` seems to work.

This is all based on experimentation though so I’m not sure how it’s supposed to work to be honest.

---

<div class="post-metadata">

**Author:** ![alha02](https://avatars.discourse-cdn.com/v4/letter/a/6a8cbe/32.png) [@alha02](https://discourse.julialang.org/u/alha02)\
**Post date:** [June 29, 2017, 9:13pm UTC](https://discourse.julialang.org/t/everywhere-using-mylib-fails-when-procs-on-remote-node/4515/8 "2017-06-29T21:13:37Z")

</div>

Strange, I can’t get it working like that. Trying to load DataFrames e.g.:

```julia
julia> using DataFrames
INFO: Precompiling module DataFrames.

julia> addprocs([("user@remote_host", :auto)])
16-element Array{Int64,1}:
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17

julia> @everywhere using DataFrames
ERROR: On worker 2:
can only precompile from node 1
compilecache at ./loading.jl:670
_require at ./loading.jl:456
require at ./loading.jl:398
eval at ./boot.jl:235
eval_ew_expr at ./distributed/macros.jl:116
#106 at ./distributed/process_messages.jl:268 [inlined]
run_work_thunk at ./distributed/process_messages.jl:56
macro expansion at ./distributed/process_messages.jl:268 [inlined]
#105 at ./event.jl:73
#remotecall_fetch#141(::Array{Any,1}, ::Function, ::Function, ::Base.Distributed.Worker, ::Expr, ::Vararg{Expr,N} where N) at ./distributed/remotecall.jl:354
remotecall_fetch(::Function, ::Base.Distributed.Worker, ::Expr, ::Vararg{Expr,N} where N) at ./distributed/remotecall.jl:346
#remotecall_fetch#144(::Array{Any,1}, ::Function, ::Function, ::Int64, ::Expr, ::Vararg{Expr,N} where N) at ./distributed/remotecall.jl:367
remotecall_fetch(::Function, ::Int64, ::Expr, ::Vararg{Expr,N} where N) at ./distributed/remotecall.jl:367
(::##1#3)() at ./distributed/macros.jl:102

...and 15 more exception(s).

Stacktrace:
 [1] sync_end() at ./task.jl:287
 [2] macro expansion at ./distributed/macros.jl:112 [inlined]
 [3] anonymous at ./<missing>:?

```

And neither ssh nor tcp seems to have any problems between the two nodes…