@everywhere using FFTW produces "Package FFTW not found in current path"

Specific Domains Julia at Scale
1

[This issue has been discussed in various places; unfortunately none of the proposed fixes seems to work for me].

When executing @everywhere CertainPackages I get an error. As suggested by several I start with:

using Distributed
addprocs(2)
@everywhere using Pkg
@everywhere Pkg.activate(“.”)

I get no complaint with

@everywhere using LinearAlgebra

but when executing

@everywhere using FFTW
@everywhere using SpecialFunctions

I get a message asking me to install FFTW and SpecialFunctions for workers != 1. See example below. Is there a workaround?

-Eric

InterruptException:

On worker 2:

ArgumentError: Package FFTW not found in current path:

  • Run import Pkg; Pkg.add("FFTW") to install the FFTW package.

require at .\loading.jl:823
eval at .\boot.jl:328
#116 at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Distributed\src\process_messages.jl:276
run_work_thunk at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Distributed\src\process_messages.jl:56
run_work_thunk at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Distributed\src\process_messages.jl:65
#102 at .\task.jl:259
#remotecall_wait#154(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Distributed.Worker, ::Module, ::Vararg{Any,N} where N) at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Distributed\src\remotecall.jl:421
remotecall_wait(::Function, ::Distributed.Worker, ::Module, ::Vararg{Any,N} where N) at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Distributed\src\remotecall.jl:412
#remotecall_wait#157(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Int64, ::Module, ::Vararg{Any,N} where N) at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Distributed\src\remotecall.jl:433
remotecall_wait(::Function, ::Int64, ::Module, ::Vararg{Any,N} where N) at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Distributed\src\remotecall.jl:433
(::getfield(Distributed, Symbol(“##163#165”)){Module,Expr})() at .\task.jl:259

1 Like
2

Sorry for suggesting the obvious, but have you tried:
@everywhere Pkg.add(“FFTW”)
@everywhere Pkg.add(“SpecialFunctions”)

I just now tried and on my system, the using FFTW worked without complains. The using SpecialFunctions complained, but worked after the distributed install (I would guess you typically don’t even need a distributed install and a single local Pkg.add should work, unless your running this on some cluster setup).

Maybe a

@everywhere Pkg.status()

will give some additional information, although it isn’t easy to decode the output due to arriving async.

3

Continuing the discussion from @everywhere using FFTW produces "Package FFTW not found in current path":

Thanks for the suggestion. Unfortunately, it did not work. See the output below and the ERROR midway through the message when attempting to install on worker 3 (I added 2 procs prior to doing this operation). Note that I am using a 6 core Windows laptop - this is not an attempt to use a cluster – and I am running v 1.1.0.

It seems there should be a simpler solution.

julia> @everywhere using Pkg

julia> @everywhere Pkg.add(“FFTW”)
Resolving package versions…
Updating C:\Users\emichiel\.juliapro\JuliaPro_v1.1.0.1\environments\v1.1\Project.toml
[no changes]
Updating C:\Users\emichiel\.juliapro\JuliaPro_v1.1.0.1\environments\v1.1\Manifest.toml
[no changes]
From worker 3: Cloning default registries into C:\Users\emichiel\.julia
From worker 2: Cloning default registries into C:\Users\emichiel\.julia
From worker 3: Cloning registry from “https://pkg.juliacomputing.com/registry/General”
From worker 2: Cloning registry from “https://pkg.juliacomputing.com/registry/General”
Added registry General to C:\Users\emichiel\.julia\registries\General
From worker 2: Resolving package versions…
From worker 2: Installed Reexport ─────── v0.2.0
From worker 2: Installed FFTW ─────────── v0.2.4
From worker 2: Installed Compat ───────── v2.1.0
From worker 2: Installed VersionParsing ─ v1.1.3
From worker 2: Installed AbstractFFTs ─── v0.4.1
From worker 2: Installed Conda ────────── v1.3.0
From worker 2: Installed JSON ─────────── v0.20.0
From worker 2: Updating C:\Users\emichiel\.julia\environments\v1.1\Project.toml
From worker 2: [7a1cc6ca] + FFTW v0.2.4
From worker 2: Updating C:\Users\emichiel\.julia\environments\v1.1\Manifest.toml
From worker 2: [621f4979] + AbstractFFTs v0.4.1
From worker 2: [34da2185] + Compat v2.1.0
From worker 2: [8f4d0f93] + Conda v1.3.0
From worker 2: [7a1cc6ca] + FFTW v0.2.4
From worker 2: [682c06a0] + JSON v0.20.0
From worker 2: [189a3867] + Reexport v0.2.0
From worker 2: [81def892] + VersionParsing v1.1.3
From worker 2: [2a0f44e3] + Base64
From worker 2: [b99e7846] + BinaryProvider
From worker 2: [864e158e] + CredentialsHandler
From worker 2: [ade2ca70] + Dates
From worker 2: [8bb1440f] + DelimitedFiles
From worker 2: [8ba89e20] + Distributed
From worker 2: [cd3eb016] + HTTP
From worker 2: [83e8ac13] + IniFile
From worker 2: [b77e0a4c] + InteractiveUtils
From worker 2: [76f85450] + LibGit2
From worker 2: [8f399da3] + Libdl
From worker 2: [37e2e46d] + LinearAlgebra
From worker 2: [56ddb016] + Logging
From worker 2: [d6f4376e] + Markdown
From worker 2: [739be429] + MbedTLS
From worker 2: [a63ad114] + Mmap
From worker 2: [44cfe95a] + Pkg
From worker 2: [de0858da] + Printf
From worker 2: [3fa0cd96] + REPL
From worker 2: [9a3f8284] + Random
From worker 2: [ea8e919c] + SHA
From worker 2: [9e88b42a] + Serialization
From worker 2: [1a1011a3] + SharedArrays
From worker 2: [6462fe0b] + Sockets
From worker 2: [2f01184e] + SparseArrays
From worker 2: [10745b16] + Statistics
From worker 2: [9d418dce] + TOML
From worker 2: [8dfed614] + Test
From worker 2: [cf7118a7] + UUIDs
From worker 2: [4ec0a83e] + Unicode
From worker 2: Building Conda → C:\Users\emichiel\.julia\packages\Conda\kLXeC\deps\build.log
From worker 2: Building FFTW ─→ C:\Users\emichiel\.julia\packages\FFTW\p7sLQ\deps\build.log
ERROR: On worker 3:
SystemError (with C:\Users\emichiel.julia\registries\General): mkdir: File exists
#systemerror#43 at .\error.jl:134
#systemerror at .\none:0 [inlined]
#mkdir#7 at .\file.jl:167
mkdir at .\file.jl:162 [inlined]
#cptree#11 at .\file.jl:303
#cptree at .\none:0 [inlined]
#cp#12 at .\file.jl:334
cp at .\file.jl:330
clone_or_cp_registries at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Pkg\src\Types.jl:1140
clone_or_cp_registries at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Pkg\src\Types.jl:1096 [inlined]
clone_default_registries at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Pkg\src\Types.jl:1046
find_registered! at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Pkg\src\Types.jl:1323
registry_resolve! at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Pkg\src\Types.jl:950
#add_or_develop#15 at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Pkg\src\API.jl:47
#add_or_develop at .\none:0 [inlined]
#add_or_develop#14 at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Pkg\src\API.jl:31 [inlined]
#add_or_develop at .\none:0 [inlined]
#add_or_develop#13 at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Pkg\src\API.jl:29 [inlined]
#add_or_develop at .\none:0 [inlined]
#add_or_develop#12 at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Pkg\src\API.jl:28
#add_or_develop at .\none:0 [inlined]
#add#20 at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Pkg\src\API.jl:59 [inlined]
add at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Pkg\src\API.jl:59
top-level scope at none:0
eval at .\boot.jl:328
#116 at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Distributed\src\process_messages.jl:276
run_work_thunk at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Distributed\src\process_messages.jl:56
run_work_thunk at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Distributed\src\process_messages.jl:65
#102 at .\task.jl:259
#remotecall_wait#154(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Distributed.Worker, ::Module, ::Vararg{Any,N} where N) at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Distributed\src\remotecall.jl:421
remotecall_wait(::Function, ::Distributed.Worker, ::Module, ::Vararg{Any,N} where N) at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Distributed\src\remotecall.jl:412
#remotecall_wait#157(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Int64, ::Module, ::Vararg{Any,N} where N) at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Distributed\src\remotecall.jl:433
remotecall_wait(::Function, ::Int64, ::Module, ::Vararg{Any,N} where N) at C:\Users\julia\AppData\Local\Julia-1.1.0\share\julia\stdlib\v1.1\Distributed\src\remotecall.jl:433
(::getfield(Distributed, Symbol(“##163#165”)){Module,Expr})() at .\task.jl:259
…

4

The @everywhere Pkg.add is probably not such a good idea, since all the workers will do the download and install over each other, from there the error you saw as some worker had already created the directory.
The reason I suggested doing the @everywhere Pkg.add, is that on a cluster all the cluster nodes would then attempt to install and even if they don’t share the project.toml file explicitly, they should then hopefully be able to see the package.
On a normal single machine, doing the Pkg.add once should install and it should be available for everyone using that some location as their project.toml (which you try to set using the Pkg.activate).

My other concern would be that for some reason the workers are not all using the same directory, if they are still in a default directory after being startup up, while your main REPL is in a different directory. I would be tempted to suggest trying to change the Pkg.activate to contain the full path to the directory your working in to try to force them even more explicitly. I doubt it would make the difference, but otherwise I’m at a loss of what to suggest.