I’m trying to do some simulations in Julia using a large cluster, but when I try to load packages, I get ArgumentError: invalid checksum in cache file /home/.../. Here is how I am loading my packages:
using Distributed
[add procs on several nodes]
using A, B, C
@everywhere using A, B, C
I am almost sure that this isn’t the right way to do this, and the error is probably hiding here somewhere. How have other people successfully loaded packages onto multiple nodes with the new package system?
p.s. I promise that I’ll make a PR to update the docs once I get this sorted out.
Ok, the cache issues were caused by the @everywhere using ... line, which makes sense. I didn’t need to do that in previous versions of julia. But, now I’ve got another problem. Here is a working example:
julia> using Distributed
julia> addprocs(["node-01"])
1-element Array{Int64,1}:
2
julia> using Dates
julia> now()
2018-10-16T11:00:25.748
julia> remotecall_fetch(now, 2)
2018-10-16T11:00:45.624
julia> @everywhere get_now() = now()
julia> get_now()
2018-10-16T11:01:16.79
julia> remotecall_fetch(get_now, 2)
ERROR: On worker 2:
UndefVarError: now not defined
...
julia> @everywhere get_now2() = Dates.now()
julia> get_now2()
2018-10-16T11:01:47.398
julia> remotecall_fetch(get_now2, 2)
ERROR: On worker 2:
UndefVarError: Dates not defined
...
Note that the error messages you are receiving are the same as when trying to execute the commands locally.
In a fresh REPL:
Julia-1.0.1> now()
ERROR: UndefVarError: now not defined
Stacktrace:
[1] top-level scope at none:0
Julia-1.0.1> Dates.now()
ERROR: UndefVarError: Dates not defined
Stacktrace:
[1] top-level scope at none:0
Julia-1.0.1> using Dates
Julia-1.0.1> Dates.now()
2018-10-17T15:22:17.274
Good point. But I would like to avoid @everywhere using because it was leading to issues with precompilation (not for Dates, but for other packages that aren’t complied into the system image).
I think that this worked in past versions of Julia, but now I’m starting to doubt myself…