Installing LightGraph on Julia 0.7 master fails

ufechner7 · July 26, 2018, 3:37pm

Installing LightGraph on Julia 0.7 fails. I created a ticket (Precompiling LightGraph fails on Julia 0.7 · Issue #942 · sbromberger/LightGraphs.jl · GitHub), but it was immediately closed with the remark this would be a problem of Juliy 0.7 and arpack. The error message is giving a different reason, though:

ERROR: LoadError: ArgumentError: Package LightGraphs does not have IterativeEigensolvers in its dependencies:
- If you have LightGraphs checked out for development and have
  added IterativeEigensolvers as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with LightGraphs
Stacktrace:
 [1] require(::Module, ::Symbol) at ./loading.jl:832
 [2] include at ./boot.jl:317 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1050
 [4] include(::Module, ::String) at ./sysimg.jl:29
 [5] top-level scope at none:0
 [6] eval at ./boot.jl:319 [inlined]
 [7] eval(::Expr) at ./client.jl:394
 [8] top-level scope at ./none:3
in expression starting at /home/ufechner/.julia/packages/LightGraphs/5nPh/src/LightGraphs.jl:9
ERROR: LoadError: LoadError: Failed to precompile LightGraphs to /home/ufechner/.julia/compiled/v0.7/LightGraphs/Xm08.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] macro expansion at ./logging.jl:298 [inlined]
 [3] compilecache(::Base.PkgId) at ./loading.jl:1189
 [4] _require(::Base.PkgId) at ./loading.jl:958
 [5] require(::Base.PkgId) at ./loading.jl:854
 [6] macro expansion at ./logging.jl:299 [inlined]
 [7] require(::Module, ::Symbol) at ./loading.jl:836
 [8] include at ./boot.jl:317 [inlined]
 [9] include_relative(::Module, ::String) at ./loading.jl:1050
 [10] include(::Module, ::String) at ./sysimg.jl:29
 [11] include(::String) at ./client.jl:393
 [12] top-level scope at none:0
 [13] include at ./boot.jl:317 [inlined]
 [14] include_relative(::Module, ::String) at ./loading.jl:1050
 [15] include(::Module, ::String) at ./sysimg.jl:29
 [16] include(::String) at ./client.jl:393
 [17] top-level scope at none:0

I am not interested in a quick fix, but I want to understand the problem and help to find a clean solution. I must install Julia from source (sometimes), and I need to be able to test the packages, that I am using, in this case JuMP. When I try to test JuMP I run into this problem.

Mason · July 26, 2018, 3:54pm

I think that this is because of the Arpack issue that @anon94023334 was so frustrated with.

kristoffer.carlsson · July 26, 2018, 4:49pm

It’s not. That was a build problem.

Mason · July 26, 2018, 4:56pm

But doesn’t IterativeEigensolvers no longer exist?

ufechner7 · July 26, 2018, 7:18pm

So as I see it now, there are two different issues:

the dependencies of LightGraph are not defined correctly
LightGraph should be dependent on Arpack.jl, but not on IterativeSolvers.jl, but building Arpack.jl might fail under curtain conditions

Could that be correct? If yes, how to verify this assumption?
I am working on Ubuntu 16.04, in cause it should matter.

How can I check, on which packages LightGraph.jl does depend?

ufechner7 · July 26, 2018, 7:31pm

Ok, I am one step further. LightGraph depends on ItereativeSolvers.jl. LightGraph#master depends on Arpack.jl, and building that is a challenge. I will try.

ufechner7 · July 26, 2018, 7:34pm

Trying to build Arpack gives the following error (not a good error message):

julia> Pkg.build("Arpack")
WARNING: Base.Pkg is deprecated, run `using Pkg` instead
 in module Main
  Building Arpack → `~/.julia/packages/Arpack/iUWc/deps/build.log`
┌ Error: Error building `Arpack`: 
│ ┌ Warning: `wait(t::Task)` is deprecated, use `fetch(t)` instead.
│ │   caller = macro expansion at OutputCollector.jl:62 [inlined]
│ └ @ Core ~/.julia/packages/BinaryProvider/mWAR/src/OutputCollector.jl:62
│ ┌ Warning: `wait(t::Task)` is deprecated, use `fetch(t)` instead.
│ │   caller = wait(::OutputCollector) at OutputCollector.jl:160
│ └ @ BinaryProvider ~/.julia/packages/BinaryProvider/mWAR/src/OutputCollector.jl:160
│ ┌ Warning: `wait(t::Task)` is deprecated, use `fetch(t)` instead.
│ │   caller = wait(::OutputCollector) at OutputCollector.jl:161
│ └ @ BinaryProvider ~/.julia/packages/BinaryProvider/mWAR/src/OutputCollector.jl:161
│ ERROR: LoadError: LibraryProduct(nothing, ["libarpack"], :libarpack, "Prefix(/home/ufechner/.julia/packages/Arpack/iUWc/deps/usr)") is not satisfied, cannot generate deps.jl!
│ Stacktrace:
│  [1] error(::String) at ./error.jl:33
│  [2] #write_deps_file#134(::Bool, ::Function, ::String, ::Array{LibraryProduct,1}) at /home/ufechner/.julia/packages/BinaryProvider/mWAR/src/Products.jl:392
│  [3] write_deps_file(::String, ::Array{LibraryProduct,1}) at /home/ufechner/.julia/packages/BinaryProvider/mWAR/src/Products.jl:379
│  [4] top-level scope at none:0
│  [5] include at ./boot.jl:317 [inlined]
│  [6] include_relative(::Module, ::String) at ./loading.jl:1050
│  [7] include(::Module, ::String) at ./sysimg.jl:29
│  [8] include(::String) at ./client.jl:393
│  [9] top-level scope at none:0
│ in expression starting at /home/ufechner/.julia/packages/Arpack/iUWc/deps/build.jl:40
└ @ Pkg.Operations ~/julia/usr/share/julia/stdlib/v0.7/Pkg/src/Operations.jl:100

ufechner7 · July 26, 2018, 7:51pm

So would you suggest to install gcc 7 on my Ubuntu, or would that create more problems than it would solve?

Keno · July 26, 2018, 8:00pm

You can download the official binary build of julia-beta2, which should work fine for now. @staticfloat is working on improving the situation with binary source builds from older versions of GCC on Linux.

ufechner7 · July 26, 2018, 8:10pm

Downloading an official binary is not what I am interested in. I want to know how to make Arpack.jl work when compiling Julia from source. Mainly, because I test to which degree the regressions of Julia master with respect to 0.6 are getting fixed, and Julia master is already much better than Julia-beta2.

At least until the official 1.0 release.

Keno · July 26, 2018, 8:11pm

Building with gcc7 would work fine.

anon94023334 · July 26, 2018, 10:22pm

Ref

https://github.com/JuliaGraphs/LightGraphs.jl/issues/941
https://github.com/JuliaGraphs/LightGraphs.jl/issues/942
https://github.com/JuliaLinearAlgebra/Arpack.jl/issues/5

anon94023334 · July 26, 2018, 10:23pm

It’s not. That was a build problem.

I believe this is the same issue. How is it different?

ufechner7 · July 27, 2018, 8:32am

The error messages I posted, you get them when you try to install the last released version of LightGraph on 0.7. If you try to precompile the master version the error messages are different:

(v0.7) pkg> add LightGraphs#master
  Updating git-repo `https://github.com/JuliaGraphs/LightGraphs.jl.git`
┌ Warning: package LightGraphs at /tmp/tmpDfkjiD will need to have a [Julia]Project.toml file in the future
└ @ Pkg.Types ~/julia/usr/share/julia/stdlib/v0.7/Pkg/src/Types.jl:619
 Resolving package versions...
  Updating `~/.julia/environments/v0.7/Project.toml`
  [093fc24a] ~ LightGraphs v0.13.1+ #master (https://github.com/JuliaGraphs/LightGraphs.jl.git)
  Updating `~/.julia/environments/v0.7/Manifest.toml`
  [093fc24a] ~ LightGraphs v0.13.1+ #master (https://github.com/JuliaGraphs/LightGraphs.jl.git)

(v0.7) pkg> precompile
Precompiling project...
Precompiling LightGraphs [1 of 2]
ERROR: LoadError: No deps.jl file could be found. Please try running Pkg.build("Arpack").
Currently, the build command might fail when Julia has been built from source
and the recommendation is to use the official binaries from julialang.org.
For more info see https://github.com/JuliaLinearAlgebra/Arpack.jl/issues/5.

Stacktrace:
 [1] top-level scope at /home/ufechner/.julia/packages/Arpack/iUWc/src/Arpack.jl:15
 [2] include at ./boot.jl:317 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1050
 [4] include(::Module, ::String) at ./sysimg.jl:29
 [5] top-level scope at none:0
 [6] eval at ./boot.jl:319 [inlined]
 [7] eval(::Expr) at ./client.jl:394
 [8] top-level scope at ./none:3
in expression starting at /home/ufechner/.julia/packages/Arpack/iUWc/src/Arpack.jl:12
ERROR: LoadError: Failed to precompile Arpack to /home/ufechner/.julia/compiled/v0.7/Arpack/X5VZ.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] macro expansion at ./logging.jl:298 [inlined]
 [3] compilecache(::Base.PkgId) at ./loading.jl:1189
 [4] macro expansion at ./logging.jl:298 [inlined]
 [5] _require(::Base.PkgId) at ./loading.jl:945
 [6] require(::Base.PkgId) at ./loading.jl:854
 [7] macro expansion at ./logging.jl:299 [inlined]
 [8] require(::Module, ::Symbol) at ./loading.jl:836
 [9] include at ./boot.jl:317 [inlined]
 [10] include_relative(::Module, ::String) at ./loading.jl:1050
 [11] include(::Module, ::String) at ./sysimg.jl:29
 [12] top-level scope at none:0
 [13] eval at ./boot.jl:319 [inlined]
 [14] eval(::Expr) at ./client.jl:394
 [15] top-level scope at ./none:3
in expression starting at /home/ufechner/.julia/packages/LightGraphs/h41J/src/LightGraphs.jl:7
ERROR: Failed to precompile LightGraphs to /home/ufechner/.julia/compiled/v0.7/LightGraphs/Xm08.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] macro expansion at ./logging.jl:298 [inlined]
 [3] compilecache(::Base.PkgId) at ./loading.jl:1189
 [4] _require(::Base.PkgId) at ./loading.jl:958
 [5] require(::Base.PkgId) at ./loading.jl:854
 [6] macro expansion at ./logging.jl:299 [inlined]
 [7] require(::Module, ::Symbol) at ./loading.jl:836
Precompiling Arpack [2 of 2]
ERROR: LoadError: No deps.jl file could be found. Please try running Pkg.build("Arpack").
Currently, the build command might fail when Julia has been built from source
and the recommendation is to use the official binaries from julialang.org.
For more info see https://github.com/JuliaLinearAlgebra/Arpack.jl/issues/5.

Stacktrace:
 [1] top-level scope at /home/ufechner/.julia/packages/Arpack/iUWc/src/Arpack.jl:15
 [2] include at ./boot.jl:317 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1050
 [4] include(::Module, ::String) at ./sysimg.jl:29
 [5] top-level scope at none:0
 [6] eval at ./boot.jl:319 [inlined]
 [7] eval(::Expr) at ./client.jl:394
 [8] top-level scope at ./none:3
in expression starting at /home/ufechner/.julia/packages/Arpack/iUWc/src/Arpack.jl:12
ERROR: Failed to precompile Arpack to /home/ufechner/.julia/compiled/v0.7/Arpack/X5VZ.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] macro expansion at ./logging.jl:298 [inlined]
 [3] compilecache(::Base.PkgId) at ./loading.jl:1189
 [4] _require(::Base.PkgId) at ./loading.jl:958
 [5] require(::Base.PkgId) at ./loading.jl:854
 [6] macro expansion at ./logging.jl:299 [inlined]
 [7] require(::Module, ::Symbol) at ./loading.jl:836

anon94023334 · July 27, 2018, 2:55pm

To be clear, and as mentioned in the Github issue, you get them when you try to install LightGraphs 0.13.1 on a master after the move of eigs out from IterativeSovlers (which was in stdlib) to Arpack (which is third-party and requires an explicit entry in REQUIRES). Unfortunately, that change to master happened very shortly after 0.13.1 was released, and because of the Arpack build issue (see references above) we haven’t yet released a new version of LightGraphs that takes advantage of Arpack.

Please also note that we only “officially support” versions of LightGraphs that run on official releases of Julia. In this case, that means Julia 0.6.x and LightGraphs 0.12.x.

(“officially support” is a bit of a misnomer for a community-driven OSS project, but what it means is that this is where our effort is spent making sure things work properly.)

ufechner7 · July 27, 2018, 6:35pm

All I want in the moment is a version of JuMP that is working on 0.7 for me. To be able to run the tests of JuMP I need LightGraphs. So I am kind of stuck: I cannot find out if the fact the my code fails on 0.7 is due to problems in my code or problems in JuMP. I still plan to try to install LightGraphs using gcc 7.x . Didn’t find the time to do this yet.

anon94023334 · July 27, 2018, 6:38pm

Please report back with your success/failure using gcc7. That may be the best we can do at this point for people (like you and me) who can’t use the nightly builds. (I won’t be able to take advantage of the gcc7 fix even if it works, but if it helps most others who are in the same position, it’s a positive step.)

ufechner7 · July 27, 2018, 8:34pm

I can already report: Installing Arpack on 0.7 works on Ubuntu 16.04 with gcc 7.3
Installing LightGraphs#master also works, but it would be nice if installing LightGraphs (released version) would also work, which is failing.

Sitting in a Cafe in Den Haag at 34 degrees Celsius and compiling: Not nice.

Update:
Testing JuMP#master actually fails, but that is a different issue. I will report that in the JuMP repo.