"@everywhere using" ERROR for worker 2


I’m trying to use parallel processing on Julia 1.0.1 and found the following issue: the new worker is not able to find packages I already installed. Please help me do this in the right way.

In the example below, I first showed that “using DataFrames” works fine, since DataFrames has been installed. But after adding a process, “@everywhere using DataFrames” fails because worker 2 (the newly added worker) cannot find DataFrames.

julia> using DataFrames

julia> using Distributed

julia> addprocs(1)
1-element Array{Int64,1}:

julia> @everywhere using DataFrames
ERROR: On worker 2:
ArgumentError: Package DataFrames not found in current path:
- Run `import Pkg; Pkg.add("DataFrames")` to install the DataFrames package.

require at .\loading.jl:820
eval at .\boot.jl:319
#116 at C:\Users\julia\AppData\Local\Julia-1.0.1\share\julia\stdlib\v1.0\Distributed\src\process_mes
run_work_thunk at C:\Users\julia\AppData\Local\Julia-1.0.1\share\julia\stdlib\v1.0\Distributed\src\p
run_work_thunk at C:\Users\julia\AppData\Local\Julia-1.0.1\share\julia\stdlib\v1.0\Distributed\src\p
#102 at .\task.jl:259
#remotecall_wait#154(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Funct
ion, ::Function, ::Distributed.Worker, ::Module, ::Vararg{Any,N} where N) at C:\Users\julia\AppData\
remotecall_wait(::Function, ::Distributed.Worker, ::Module, ::Vararg{Any,N} where N) at C:\Users\jul
#remotecall_wait#157(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Funct
ion, ::Function, ::Int64, ::Module, ::Vararg{Any,N} where N) at C:\Users\julia\AppData\Local\Julia-1
remotecall_wait(::Function, ::Int64, ::Module, ::Vararg{Any,N} where N) at C:\Users\julia\AppData\Lo
(::getfield(Distributed, Symbol("##163#165")){Module,Expr})() at .\task.jl:259
 [1] sync_end(::Array{Any,1}) at .\task.jl:226
 [2] remotecall_eval(::Module, ::Array{Int64,1}, ::Expr) at C:\Users\julia\AppData\Local\Julia-1.0.1
 [3] top-level scope at C:\Users\julia\AppData\Local\Julia-1.0.1\share\julia\stdlib\v1.0\Distributed


Are you using a different environment than the default (e.g. ] activate . or starting with julia --project)? If so see this discussion.

Hi I am seeing the same error with exactly the same usage as the one of @Zhong_Pan, and so would like to revive this topic.

The simplest code to demonstrate the error is:

using Distributed
@everywhere using DataFrames

I’m running on JuliaPro 1.1.1 (tried the latest 1.2 and it won’t help) on Windows 8.

It’s not a different environment. And the discussion you mentioned seems to concern only user’s own package but not a curated package such as DataFrames.

I tried a lot of things and none of them seems working. I’m under the impression that it is because packages in v1.0+ are now installed “locally” on userprofile path instead of a global C: drive location.

First, obviously I’ve installed DataFrames and this line of code will run okay:

using DataFrames

Then, instead of DataFrames, a standard package such as HTTP will work:

@everywhere using HTTP

Reading through past topics but don’t seem to have find a solution.

This discussion suggests the installation wasn’t clean but I have already uninstalled and re-installed several times.

This one seems to suggest JuliaPro won’t work with distribution - I hope it’s not true and there is a solution.