Packages imported in IJulia throw errors that are missing when imported in REPL

Qfl3x · July 7, 2020, 5:35pm

Hello. When using packages from within IJulia, I get error messages of the type:

┌ Info: Precompiling Zygote [e88e6eb3-aa80-5325-afca-941959d7151f]
└ @ Base loading.jl:1260
┌ Warning: Package Zygote does not have ArrayLayouts in its dependencies:
│ - If you have Zygote checked out for development and have
│   added ArrayLayouts as a dependency but haven't updated your primary
│   environment's manifest file, try `Pkg.resolve()`.
│ - Otherwise you may need to report an issue with Zygote
└ Loading ArrayLayouts into Zygote from project dependency, future warnings for Zygote are suppressed.
ERROR: LoadError: LoadError: ArgumentError: Package Zygote does not have AbstractFFTs in its dependencies:
- If you have Zygote checked out for development and have
  added AbstractFFTs as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with Zygote
Stacktrace:
 [1] require(::Module, ::Symbol) at ./loading.jl:905
 [2] include(::Module, ::String) at ./Base.jl:377
 [3] include(::String) at /home/qfl3x/.julia/packages/Zygote/1GXzF/src/Zygote.jl:1
 [4] top-level scope at /home/qfl3x/.julia/packages/Zygote/1GXzF/src/Zygote.jl:29
 [5] include(::Module, ::String) at ./Base.jl:377
 [6] top-level scope at none:2
 [7] eval at ./boot.jl:331 [inlined]
 [8] eval(::Expr) at ./client.jl:449
 [9] top-level scope at ./none:3
in expression starting at /home/qfl3x/.julia/packages/Zygote/1GXzF/src/lib/array.jl:1
in expression starting at /home/qfl3x/.julia/packages/Zygote/1GXzF/src/Zygote.jl:29

Another example for DiffEq:

┌ Info: Precompiling DifferentialEquations [0c46a032-eb83-5123-abaf-570d42b7fbaa]
└ @ Base loading.jl:1260
┌ Warning: Package RecursiveArrayTools does not have LinearAlgebra in its dependencies:
│ - If you have RecursiveArrayTools checked out for development and have
│   added LinearAlgebra as a dependency but haven't updated your primary
│   environment's manifest file, try `Pkg.resolve()`.
│ - Otherwise you may need to report an issue with RecursiveArrayTools
└ Loading LinearAlgebra into RecursiveArrayTools from project dependency, future warnings for RecursiveArrayTools are suppressed.
ERROR: LoadError: LoadError: ArgumentError: Package RecursiveFactorization does not have LoopVectorization in its dependencies:
- If you have RecursiveFactorization checked out for development and have
  added LoopVectorization as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with RecursiveFactorization
Stacktrace:
 [1] require(::Module, ::Symbol) at ./loading.jl:905
 [2] include(::Module, ::String) at ./Base.jl:377
 [3] include(::String) at /home/qfl3x/.julia/packages/RecursiveFactorization/wAEz3/src/RecursiveFactorization.jl:1
 [4] top-level scope at /home/qfl3x/.julia/packages/RecursiveFactorization/wAEz3/src/RecursiveFactorization.jl:3
 [5] include(::Module, ::String) at ./Base.jl:377
 [6] top-level scope at none:2
 [7] eval at ./boot.jl:331 [inlined]
 [8] eval(::Expr) at ./client.jl:449
 [9] top-level scope at ./none:3
in expression starting at /home/qfl3x/.julia/packages/RecursiveFactorization/wAEz3/src/lu.jl:1
in expression starting at /home/qfl3x/.julia/packages/RecursiveFactorization/wAEz3/src/RecursiveFactorization.jl:3
ERROR: LoadError: Failed to precompile RecursiveFactorization [f2c3362d-daeb-58d1-803e-2bc74f2840b4] to /home/qfl3x/.julia/compiled/v1.4/RecursiveFactorization/tEwl6_4K85o.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
 [3] _require(::Base.PkgId) at ./loading.jl:1029
 [4] require(::Base.PkgId) at ./loading.jl:927
 [5] require(::Module, ::Symbol) at ./loading.jl:922
 [6] include(::Module, ::String) at ./Base.jl:377
 [7] top-level scope at none:2
 [8] eval at ./boot.jl:331 [inlined]
 [9] eval(::Expr) at ./client.jl:449
 [10] top-level scope at ./none:3
in expression starting at /home/qfl3x/.julia/packages/DiffEqBase/uSSHl/src/DiffEqBase.jl:3
ERROR: LoadError: Failed to precompile DiffEqBase [2b5f629d-d688-5b77-993f-72d75c75574e] to /home/qfl3x/.julia/compiled/v1.4/DiffEqBase/oaevG_4K85o.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
 [3] _require(::Base.PkgId) at ./loading.jl:1029
 [4] require(::Base.PkgId) at ./loading.jl:927
 [5] require(::Module, ::Symbol) at ./loading.jl:922
 [6] include(::Module, ::String) at ./Base.jl:377
 [7] top-level scope at none:2
 [8] eval at ./boot.jl:331 [inlined]
 [9] eval(::Expr) at ./client.jl:449
 [10] top-level scope at ./none:3
in expression starting at /home/qfl3x/.julia/packages/DifferentialEquations/fpohE/src/DifferentialEquations.jl:5

These messages only go away if I painstakingly Pkg.add every package that’s written in there. However, this does not occure in the REPL, where only Precompilation messages appear. And yes, I am using the good version of Julia for IJulia, otherwise installing those packages won’t make the errors disappear.

This is happening for me on two machines. An Arch Linux machine with the distro’s julia and the other a Solus machine with a binary.

kristoffer.carlsson · July 7, 2020, 5:41pm

Stuff that are good to know:

What Julia version?
How did you install Julia?
Do you have multiple registries installed (in e.g. .julia/registries)?
How does your Manifest.toml file look like after adding Zygote.

Henrique_Becker · July 7, 2020, 5:50pm

This is strange. I see this kind of error only when I am executing tests or generating documentation for a package I have written, and the Manifest.toml inside test or docs is old, so I already have added new dependencies to my package, but did not update the environment of test or docs to know this.

Qfl3x · July 7, 2020, 5:53pm

julia> versioninfo()
Julia Version 1.4.2
Commit 44fa15b150* (2020-05-23 18:35 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, skylake)

The version I’m currently working with is the binary version.

Only one registry. ( General )

Zygote entry:

[[Zygote]]
deps = ["AbstractFFTs", "ArrayLayouts", "ChainRules", "FillArrays", "ForwardDiff", "Future", "IRTools", "InteractiveUtils", "LinearAlgebra", "MacroTools", "NNlib", "Random", "Requires", "Statistics", "ZygoteRules"]
git-tree-sha1 = "2e2c82549fb0414df10469082fd001e2ede8547c"
uuid = "e88e6eb3-aa80-5325-afca-941959d7151f"
version = "0.4.22"

Qfl3x · July 7, 2020, 8:46pm

UPDATE: I reset my julia install (rm -r ~/.julia) and it still happens. However, this only happens in the notebooks that I git clone’d from here

kristoffer.carlsson · July 7, 2020, 8:53pm

In the Manifest there, Zygote does not have ArrayLayouts as a dependency.

Qfl3x · July 7, 2020, 8:54pm

Isn’t it in deps ? After AbstractFFTs

kristoffer.carlsson · July 7, 2020, 9:00pm

I meant here:

https://github.com/crstnbr/JuliaOulu20/blob/1479195808f2bb4898f20007196ba4919332443e/Manifest.toml#L880

It seems it is loading a new version of Zygote but the one in the manifest has dependencies for an older one.

kristoffer.carlsson · July 7, 2020, 9:08pm

It works to load Zygote for me in that notebook:

Qfl3x · July 7, 2020, 9:09pm

Oh. I did not know about the environment. I just went in and loaded the notebooks. So now I just deleted the environment files (manifest, project, environment) and everything is working fine!