"@everywhere using" ERROR for worker 2

General Usage
1

Hi,

I’m trying to use parallel processing on Julia 1.0.1 and found the following issue: the new worker is not able to find packages I already installed. Please help me do this in the right way.

In the example below, I first showed that “using DataFrames” works fine, since DataFrames has been installed. But after adding a process, “@everywhere using DataFrames” fails because worker 2 (the newly added worker) cannot find DataFrames.

julia> using DataFrames

julia> using Distributed

julia> addprocs(1)
1-element Array{Int64,1}:
 2

julia> @everywhere using DataFrames
ERROR: On worker 2:
ArgumentError: Package DataFrames not found in current path:
- Run `import Pkg; Pkg.add("DataFrames")` to install the DataFrames package.

require at .\loading.jl:820
eval at .\boot.jl:319
#116 at C:\Users\julia\AppData\Local\Julia-1.0.1\share\julia\stdlib\v1.0\Distributed\src\process_mes
sages.jl:276
run_work_thunk at C:\Users\julia\AppData\Local\Julia-1.0.1\share\julia\stdlib\v1.0\Distributed\src\p
rocess_messages.jl:56
run_work_thunk at C:\Users\julia\AppData\Local\Julia-1.0.1\share\julia\stdlib\v1.0\Distributed\src\p
rocess_messages.jl:65
#102 at .\task.jl:259
#remotecall_wait#154(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Funct
ion, ::Function, ::Distributed.Worker, ::Module, ::Vararg{Any,N} where N) at C:\Users\julia\AppData\
Local\Julia-1.0.1\share\julia\stdlib\v1.0\Distributed\src\remotecall.jl:421
remotecall_wait(::Function, ::Distributed.Worker, ::Module, ::Vararg{Any,N} where N) at C:\Users\jul
ia\AppData\Local\Julia-1.0.1\share\julia\stdlib\v1.0\Distributed\src\remotecall.jl:412
#remotecall_wait#157(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Funct
ion, ::Function, ::Int64, ::Module, ::Vararg{Any,N} where N) at C:\Users\julia\AppData\Local\Julia-1
.0.1\share\julia\stdlib\v1.0\Distributed\src\remotecall.jl:433
remotecall_wait(::Function, ::Int64, ::Module, ::Vararg{Any,N} where N) at C:\Users\julia\AppData\Lo
cal\Julia-1.0.1\share\julia\stdlib\v1.0\Distributed\src\remotecall.jl:433
(::getfield(Distributed, Symbol("##163#165")){Module,Expr})() at .\task.jl:259
Stacktrace:
 [1] sync_end(::Array{Any,1}) at .\task.jl:226
 [2] remotecall_eval(::Module, ::Array{Int64,1}, ::Expr) at C:\Users\julia\AppData\Local\Julia-1.0.1
\share\julia\stdlib\v1.0\Distributed\src\macros.jl:207
 [3] top-level scope at C:\Users\julia\AppData\Local\Julia-1.0.1\share\julia\stdlib\v1.0\Distributed
\src\macros.jl:190

julia>
2

Are you using a different environment than the default (e.g. ] activate . or starting with julia --project)? If so see this discussion.

3

Hi I am seeing the same error with exactly the same usage as the one of @Zhong_Pan, and so would like to revive this topic.

The simplest code to demonstrate the error is:

using Distributed
addprocs(2)
@everywhere using DataFrames

I’m running on JuliaPro 1.1.1 (tried the latest 1.2 and it won’t help) on Windows 8.

It’s not a different environment. And the discussion you mentioned seems to concern only user’s own package but not a curated package such as DataFrames.

I tried a lot of things and none of them seems working. I’m under the impression that it is because packages in v1.0+ are now installed “locally” on userprofile path instead of a global C: drive location.

First, obviously I’ve installed DataFrames and this line of code will run okay:

using DataFrames

Then, instead of DataFrames, a standard package such as HTTP will work:

@everywhere using HTTP
4

Reading through past topics but don’t seem to have find a solution.

This discussion suggests the installation wasn’t clean but I have already uninstalled and re-installed several times.

This one seems to suggest JuliaPro won’t work with distribution - I hope it’s not true and there is a solution.