Mac "Resolving package versions..." takes a long time


#1

I’ve been using Julia for a while, but this has me flummoxed. At work on Arch, a package test spends minimal time in the package resolving stage, but on my Mac it drags.

I have quite a few dependencies, and I would understand it taking a while the first time…

Really a killer on actually doing development when testing takes an eon to just fail.


#2

That is interesting, I have personally never seen it be slow. What time scale are we talking about? What’s the stacktrace from a Ctrl+C? If you copy and paste the environment (Project.toml + Manifest.toml) to another system, is it still slow?


#3

Same with me on Windows - 5-7 sec on a very basic code and simple tests. Ctrl-C does not reveal much:

(LessOLS) pkg> test
   Testing LessOLS
 Resolving package versions...
ERROR: LoadError: InterruptException:
Stacktrace:
 [1] include(::String) at .\client.jl:392
 [2] top-level scope at none:0
in expression starting at D:\github\LessOLS.jl\test\runtests.jl:2
ERROR: Package LessOLS errored during testing

Also when running tests on Travis - it takes about a centrury (> 1min) to pass after “Resolving package versions” and complete tests at https://travis-ci.org/epogrebnyak/LessOLS.jl

Cross-posting at github issues


#4

For me I get:

julia> import Pkg

julia> @time Pkg.test("LessOLS")
   Testing LessOLS
 Resolving package versions...
   Testing LessOLS tests passed 
  4.706391 seconds (822.65 k allocations: 47.687 MiB, 0.25% gc time)

Note that the code takes quite a lot of time to precompile:

julia> @time using LessOLS
[ Info: Precompiling LessOLS [3e4c6e70-f926-11e8-0fa5-39915c5a2b72]
 39.171277 seconds (2.73 M allocations: 151.785 MiB, 0.40% gc time)

which is what Travis has to do before the tests are run. The whole process taking ~50 seconds on the Travis nodes doesn’t seem that odd comparing to the time I get on my laptop.


#5

Why could the precompile be very long? Or perhaps is it too broad question


#6

Seems like Distributions is slow to precompile.

julia> @time using Distributions
[ Info: Precompiling Distributions [31c24e10-a181-5473-b8eb-7969acd0382f]
 36.369072 seconds (2.67 M allocations: 148.948 MiB, 0.36% gc time)

#7

I cleaned up the compiled cache by deleteing everything at /.julia /compiled and compile time reduced, here are times before and after:

julia> @time using LessOLS
  1.934703 seconds (1.60 M allocations: 95.501 MiB, 2.34% gc time)

julia> @time using LessOLS
  0.001462 seconds (587 allocations: 31.250 KiB)

Could it be Travis had Julia installation with precompiled cache? I would expect the oppisite though.


#8

I don’t believe those numbers, it should print [ Info: Precompiling LessOLS. Did you restart julia in between?


#9

You are right, I did measure without exiting the session. After exiting the session the precompile time is similar to what you have:

julia> @time using LessOLS
[ Info: Precompiling LessOLS [3e4c6e70-f926-11e8-0fa5-39915c5a2b72]
 57.375232 seconds (2.92 M allocations: 159.585 MiB, 0.20% gc time)

#10

Most of 57 sec time is precompilation of Distributions:

julia> @time using Distributions
[ Info: Precompiling Distributions [31c24e10-a181-5473-b8eb-7969acd0382f]
 50.983350 seconds (2.91 M allocations: 159.384 MiB, 0.21% gc time)

#11

I wish there was a way around precompiling dependencies for local tests - waiting for cycles of 10s sec on trivial tests is so discouraging for development.


#12

You can always just do include("~/.julia/dev/MyPackage/tests/runtest.jl") with Revise.jl to avoid recompiling dependencies after every time you change something.