Custom Modules with Distributed

Specific Domains Julia at Scale
1

In 0.6.x of Julia, the following worked for me:

modulepath = "/home/me/code/my_modules";
addprocs(4);
push!(LOAD_PATH,modulepath);
@everywhere using MyModule
# rest of my code

In the above, MyModule is some custom module I’ve written that I want to pull into the current script. This worked without a problem before. I modified it for 0.7.0 to be

using Distributed
modulepath = "/home/me/code/my_modules";
addprocs(4);
push!(LOAD_PATH,modulepath);
@everywhere using MyModule
# rest of my code

and now I get the error

ERROR: On worker 2:
ArgumentError: Package MyModule not found in current path:
- Run `Pkg.add("MyModule")` to install the MyModule package.

require at ./loading.jl:817
eval at ./boot.jl:319
#116 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Distributed/src/process_messages.jl:276
run_work_thunk at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Distributed/src/process_messages.jl:56
run_work_thunk at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Distributed/src/process_messages.jl:65
#102 at ./task.jl:262
#remotecall_wait#154(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Distributed.Worker, ::Module, ::Vararg{Any,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Distributed/src/remotecall.jl:407
remotecall_wait(::Function, ::Distributed.Worker, ::Module, ::Vararg{Any,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Distributed/src/remotecall.jl:398
#remotecall_wait#157(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Int64, ::Module, ::Vararg{Any,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Distributed/src/remotecall.jl:419
remotecall_wait(::Function, ::Int64, ::Module, ::Vararg{Any,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Distributed/src/remotecall.jl:419
(::getfield(Distributed, Symbol("##163#165")){Module,Expr})() at ./task.jl:262

...and 15 more exception(s).

Stacktrace:
 [1] sync_end(::Array{Any,1}) at ./task.jl:229
 [2] remotecall_eval(::Module, ::Array{Int64,1}, ::Expr) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Distributed/src/macros.jl:207
 [3] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v0.7/Distributed/src/macros.jl:190
2

@everywhere push!(LOAD_PATH,modulepath);

Or:

@everywhere begin
    modulepath = "/home/me/code/my_modules";
    push!(LOAD_PATH,modulepath);
    using MyModule
end

Should do the trick.