Help with failing package compilaton of AbstractFFTs

General Usage
, , , ,
1

Hi I’m trying to install CUDA.jl, but its dependency AbstractFFTs keeps failing. Here is an output from the session. Is there any way to debug it? Is the problem on my end? Neither Julia 1.8, 1.7, nor 1.6 works.

$ julia --project
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.8.0 (2022-08-17)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(jltest) pkg> add AbstractFFTs
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
    Updating `~/jltest/Project.toml`
  [621f4979] + AbstractFFTs v1.2.1
    Updating `~/jltest/Manifest.toml`
  [621f4979] + AbstractFFTs v1.2.1
  [d360d2e6] + ChainRulesCore v1.15.6
  [34da2185] + Compat v4.3.0
  [56f22d72] + Artifacts
  [ade2ca70] + Dates
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [de0858da] + Printf
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [2f01184e] + SparseArrays
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v0.5.2+0
  [4536629a] + OpenBLAS_jll v0.3.20+0
  [8e850b90] + libblastrampoline_jll v5.1.1+0
Precompiling project...
  ✗ AbstractFFTs
  0 dependencies successfully precompiled in 2 seconds. 5 already precompiled.
  2 dependencies errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the packages

(jltest) pkg> precompile
Precompiling project...
  ✗ AbstractFFTs
  0 dependencies successfully precompiled in 2 seconds. 5 already precompiled.

ERROR: The following 1 direct dependency failed to precompile:

AbstractFFTs [621f4979-c628-5d54-868e-fcf4e3e8185c]

Failed to precompile AbstractFFTs [621f4979-c628-5d54-868e-fcf4e3e8185c] to /home/votroto1/.julia/compiled/v1.8/AbstractFFTs/jl_JUgasr.
ERROR: LoadError: ArgumentError: Package ChainRulesCore [d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Stacktrace:
 [1] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1306
 [2] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1200
 [3] macro expansion
   @ ./loading.jl:1180 [inlined]
 [4] macro expansion
   @ ./lock.jl:223 [inlined]
 [5] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1144
 [6] include
   @ ./Base.jl:419 [inlined]
 [7] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
   @ Base ./loading.jl:1554
 [8] top-level scope
   @ stdin:1
in expression starting at /home/votroto1/.julia/packages/AbstractFFTs/Wg2Yf/src/AbstractFFTs.jl:1
in expression starting at stdin:1

(jltest) pkg> instantiate
  0 dependencies successfully precompiled in 0 seconds. 5 already precompiled. 1 skipped during auto due to previous errors.
  1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package

(jltest) pkg> precompile
Precompiling project...
  ✗ AbstractFFTs
  0 dependencies successfully precompiled in 2 seconds. 5 already precompiled.

ERROR: The following 1 direct dependency failed to precompile:

AbstractFFTs [621f4979-c628-5d54-868e-fcf4e3e8185c]

Failed to precompile AbstractFFTs [621f4979-c628-5d54-868e-fcf4e3e8185c] to /home/votroto1/.julia/compiled/v1.8/AbstractFFTs/jl_QO4Jje.
ERROR: LoadError: ArgumentError: Package ChainRulesCore [d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Stacktrace:
 [1] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1306
 [2] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1200
 [3] macro expansion
   @ ./loading.jl:1180 [inlined]
 [4] macro expansion
   @ ./lock.jl:223 [inlined]
 [5] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1144
 [6] include
   @ ./Base.jl:419 [inlined]
 [7] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
   @ Base ./loading.jl:1554
 [8] top-level scope
   @ stdin:1
in expression starting at /home/votroto1/.julia/packages/AbstractFFTs/Wg2Yf/src/AbstractFFTs.jl:1
in expression starting at stdin:1

and here is the system info

julia> versioninfo()
Julia Version 1.8.0
Commit 5544a0fab76 (2022-08-17 13:38 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 72 × Intel(R) Xeon(R) Gold 6150 CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake-avx512)
  Threads: 1 on 72 virtual cores
Environment:
  LD_LIBRARY_PATH = /mnt/appl/software/LAPACK/3.9.1-GCC-10.2.0/lib:/mnt/appl/software/binutils/2.35-GCCcore-10.2.0/lib:/mnt/appl/software/zlib/1.2.11-GCCcore-10.2.0/lib:/mnt/appl/software/GCCcore/10.2.0/lib64:/mnt/appl/software/GCCcore/10.2.0/lib:/mnt/appl/software/OpenBLAS/0.3.20-GCC-11.3.0/lib:/mnt/appl/software/Julia/1.8.0-linux-x86_64/lib:/mnt/appl/software/CUDAcore/11.4.1/nvvm/lib64:/mnt/appl/software/CUDAcore/11.4.1/extras/CUPTI/lib64:/mnt/appl/software/CUDAcore/11.4.1/lib:/mnt/appl/software/CUDA/11.7.0/nvvm/lib64:/mnt/appl/software/CUDA/11.7.0/extras/CUPTI/lib64:/mnt/appl/software/CUDA/11.7.0/lib

Thank You

2

There were a lot of similar issues recently, mostly connected to precompilation spinning up to much BLAS threads. It happened mostly on AMD machines, but it would seem it affects those with high core count, like yours.
Please try the newest version (1.8.2) as it resolved most of the issues reported.

3

The error message suggests

Failed to precompile AbstractFFTs [621f4979-c628-5d54-868e-fcf4e3e8185c] to /home/votroto1/.julia/compiled/v1.8/AbstractFFTs/jl_QO4Jje.
ERROR: LoadError: ArgumentError: Package ChainRulesCore [d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Have you tried running Pkg.instantiate()?

4

Of course. The highlighting on the pasted REPL session isn’t very readable, but it is right below where it recommends that.

5

Apologies, I must have missed that. Could you run

import Pkg; Pkg.precompile()

in the repl instead of precompile in the pkg mode? This sometimes provides additional information about the error.

In any case, worth checking if v1.8.2 resolves the issue.

6

Sadly, 1.8.2 does not fix the issue. and import Pkg; Pkg.precompile() provides no additional info.

(scut) pkg> add AbstractFFTs
   Resolving package versions...
    Updating `~/scut/Project.toml`
  [621f4979] + AbstractFFTs v1.2.1
    Updating `~/scut/Manifest.toml`
  [621f4979] + AbstractFFTs v1.2.1
  [d360d2e6] + ChainRulesCore v1.15.6
  [34da2185] + Compat v4.3.0
  [56f22d72] + Artifacts
  [ade2ca70] + Dates
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [de0858da] + Printf
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [2f01184e] + SparseArrays
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v0.5.2+0
  [4536629a] + OpenBLAS_jll v0.3.20+0
  [8e850b90] + libblastrampoline_jll v5.1.1+0
  0 dependencies successfully precompiled in 0 seconds. 5 already precompiled. 1 skipped during auto due to previous errors.
  1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package

julia> import Pkg; Pkg.precompile()
Precompiling project...
  ✗ AbstractFFTs
  0 dependencies successfully precompiled in 1 seconds. 5 already precompiled.

ERROR: The following 1 direct dependency failed to precompile:

AbstractFFTs [621f4979-c628-5d54-868e-fcf4e3e8185c]

Failed to precompile AbstractFFTs [621f4979-c628-5d54-868e-fcf4e3e8185c] to /home/votroto1/.julia/compiled/v1.8/AbstractFFTs/jl_hYRqhj.
ERROR: LoadError: ArgumentError: Package ChainRulesCore [d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Stacktrace:
 [1] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1306
 [2] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1200
 [3] macro expansion
   @ ./loading.jl:1180 [inlined]
 [4] macro expansion
   @ ./lock.jl:223 [inlined]
 [5] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1144
 [6] include
   @ ./Base.jl:419 [inlined]
 [7] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
   @ Base ./loading.jl:1554
 [8] top-level scope
   @ stdin:1
in expression starting at /home/votroto1/.julia/packages/AbstractFFTs/Wg2Yf/src/AbstractFFTs.jl:1
in expression starting at stdin:1
Stacktrace:
 [1] pkgerror(msg::String)
   @ Pkg.Types /mnt/appl_common/software/Julia/1.8.2-linux-x86_64/share/julia/stdlib/v1.8/Pkg/src/Types.jl:67
 [2] precompile(ctx::Pkg.Types.Context, pkgs::Vector{String}; internal_call::Bool, strict::Bool, warn_loaded::Bool, already_instantiated::Bool, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ Pkg.API /mnt/appl_common/software/Julia/1.8.2-linux-x86_64/share/julia/stdlib/v1.8/Pkg/src/API.jl:1432
 [3] precompile
   @ /mnt/appl_common/software/Julia/1.8.2-linux-x86_64/share/julia/stdlib/v1.8/Pkg/src/API.jl:1063 [inlined]
 [4] #precompile#225
   @ /mnt/appl_common/software/Julia/1.8.2-linux-x86_64/share/julia/stdlib/v1.8/Pkg/src/API.jl:1062 [inlined]
 [5] precompile (repeats 2 times)
   @ /mnt/appl_common/software/Julia/1.8.2-linux-x86_64/share/julia/stdlib/v1.8/Pkg/src/API.jl:1062 [inlined]
 [6] top-level scope
   @ REPL[10]:1

julia> Pkg.instantiate()
  0 dependencies successfully precompiled in 0 seconds. 5 already precompiled. 1 skipped during auto due to previous errors.
  1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package

(scut) pkg> instantiate
  0 dependencies successfully precompiled in 0 seconds. 5 already precompiled. 1 skipped during auto due to previous errors.
  1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package

julia> import Pkg; Pkg.precompile()
Precompiling project...
  ✗ AbstractFFTs
  0 dependencies successfully precompiled in 1 seconds. 5 already precompiled.

ERROR: The following 1 direct dependency failed to precompile:

AbstractFFTs [621f4979-c628-5d54-868e-fcf4e3e8185c]

Failed to precompile AbstractFFTs [621f4979-c628-5d54-868e-fcf4e3e8185c] to /home/votroto1/.julia/compiled/v1.8/AbstractFFTs/jl_EKZhRL.
ERROR: LoadError: ArgumentError: Package ChainRulesCore [d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Stacktrace:
 [1] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1306
 [2] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1200
 [3] macro expansion
   @ ./loading.jl:1180 [inlined]
 [4] macro expansion
   @ ./lock.jl:223 [inlined]
 [5] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1144
 [6] include
   @ ./Base.jl:419 [inlined]
 [7] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
   @ Base ./loading.jl:1554
 [8] top-level scope
   @ stdin:1
in expression starting at /home/votroto1/.julia/packages/AbstractFFTs/Wg2Yf/src/AbstractFFTs.jl:1
in expression starting at stdin:1
Stacktrace:
 [1] pkgerror(msg::String)
   @ Pkg.Types /mnt/appl_common/software/Julia/1.8.2-linux-x86_64/share/julia/stdlib/v1.8/Pkg/src/Types.jl:67
 [2] precompile(ctx::Pkg.Types.Context, pkgs::Vector{String}; internal_call::Bool, strict::Bool, warn_loaded::Bool, already_instantiated::Bool, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ Pkg.API /mnt/appl_common/software/Julia/1.8.2-linux-x86_64/share/julia/stdlib/v1.8/Pkg/src/API.jl:1432
 [3] precompile
   @ /mnt/appl_common/software/Julia/1.8.2-linux-x86_64/share/julia/stdlib/v1.8/Pkg/src/API.jl:1063 [inlined]
 [4] #precompile#225
   @ /mnt/appl_common/software/Julia/1.8.2-linux-x86_64/share/julia/stdlib/v1.8/Pkg/src/API.jl:1062 [inlined]
 [5] precompile (repeats 2 times)
   @ /mnt/appl_common/software/Julia/1.8.2-linux-x86_64/share/julia/stdlib/v1.8/Pkg/src/API.jl:1062 [inlined]
 [6] top-level scope
   @ REPL[13]:1

Not even on a different system:

julia> versioninfo()
Julia Version 1.8.2
Commit 36034abf260 (2022-09-29 15:21 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 128 × AMD EPYC 7543 32-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, znver3)
  Threads: 1 on 128 virtual cores
Environment:
  LD_LIBRARY_PATH = /mnt/appl/software/Julia/1.8.2-linux-x86_64/lib:/mnt/appl/software/CUDA/11.7.0/nvvm/lib64:/mnt/appl/software/CUDA/11.7.0/extras/CUPTI/lib64:/mnt/appl/software/CUDA/11.7.0/lib
7

This seems quite strange, could you share your project and manifest files so that I may try to reproduce this?

8

Whoops, sorry discourse did not send me a notification. There is no project or manifest file involved, sadly.
The steps to reproduce the issue are rather simple. On a fresh machine with julia 1.6-1.8 do:

  1. ]generate problem
  2. ]activate problem
  3. ]add AbstractFFTs
  4. have a problem.

:frowning: I have still not resolved it, so if you have any suggestions, I’m all ears!

9 
(@v1.8) pkg> generate problem
  Generating  project problem:
    problem\Project.toml
    problem\src\problem.jl

(@v1.8) pkg> activate problem
  Activating project at `C:\Users\unime\problem`

(problem) pkg> add AbstractFFTs
    Updating registry at `C:\Users\unime\.julia\registries\General.toml`
   Resolving package versions...
    Updating `C:\Users\unime\problem\Project.toml`
  [621f4979] + AbstractFFTs v1.2.1
    Updating `C:\Users\unime\problem\Manifest.toml`
  [621f4979] + AbstractFFTs v1.2.1
  [d360d2e6] + ChainRulesCore v1.15.6
  [34da2185] + Compat v4.3.0
  [56f22d72] + Artifacts
  [ade2ca70] + Dates
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [de0858da] + Printf
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [2f01184e] + SparseArrays
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v0.5.2+0
  [4536629a] + OpenBLAS_jll v0.3.20+0
  [8e850b90] + libblastrampoline_jll v5.1.1+0
Precompiling project...
  1 dependency successfully precompiled in 1 seconds. 6 already precompiled.

julia> versioninfo()
Julia Version 1.8.0
Commit 5544a0fab7 (2022-08-17 13:38 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 20 × 12th Gen Intel(R) Core(TM) i9-12900HK
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, goldmont)
  Threads: 1 on 20 virtual cores
10

Well, yeah, sure… I know counter-examples exist. That’s not the problem.

The problem is that it does not work on two other machines and I can’t figure out why. It could be anything, but julia just gives me “Can’t let you do that, Dave.”

I don’t own the server and those who do do not know julia. I would appreciate some help debugging the problem.

1 Like
11

I tested it on a similar machine (EPYC 7453) but on Windows and also could not replicate.

Not working on Linux regularly, so have no idea if this is normal or not:
In your example, erroring libraries point to /mnt/appl_common/software/Julia/, while Julia itself shows this: LD_LIBRARY_PATH = /mnt/appl/software/Julia/

Might there be two instances of Julia on the server, e.g. on installed by admins and one you installed yourself that interfere with each other?

2 Likes
12

Huh… did not notice that. But the folders are just a symlinked together.

Although, Cuda.jl complained about AbstractFFTs which complained about ChainRulesCore which complained about a missing source file with no more info. I’m not sure why I did not notice that before.

Installing ChainRulesCode straight from the repo worked, though. I guess I can close this, but it would be nice to understand what is going on.

1 Like