I’ve been testing out the new Docker image for Julia 1.9-beta2. I do indeed notice the FFTX improvements, however I’m also noticing that precompilation of every package now takes on average between 1.5x-2x as long. versioninfo() yields:
Julia Version 1.9.0-beta2
Commit 7daffeecb8c (2022-12-29 07:45 UTC)
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: 12 × AMD Ryzen 5 5600G with Radeon Graphics
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-14.0.6 (ORCJIT, znver3)
Threads: 1 on 12 virtual cores
Environment:
JULIA_GPG = 3673DF529D9049477F76B37566E3C7DC03D6E495
JULIA_PATH = /usr/local/julia
JULIA_VERSION = 1.9.0-beta2
Tested against the latest version:
Julia Version 1.8.4
Commit 00177ebc4fc (2022-12-23 21:32 UTC)
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: 12 × AMD Ryzen 5 5600G with Radeon Graphics
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-13.0.1 (ORCJIT, znver3)
Threads: 1 on 12 virtual cores
Environment:
JULIA_GPG = 3673DF529D9049477F76B37566E3C7DC03D6E495
JULIA_PATH = /usr/local/julia
JULIA_VERSION = 1.8.4
I’m using Podman 4.3.1 to run the containers (podman run -it --rm julia:<latest or rc>).
I’ve compiled a list of results with some larger packages (I create a new container each time to avoid using any dependencies already precompiled):
Package
Julia 1.8.4 (seconds)
Julia 1.9.0-beta2 (seconds)
UnicodePlots 3.3.1
41
84
Enzyme 0.10.13
14
46
Unitful 1.12.2
15
32
Geostats 0.36.5
53
67
CSV 0.10.8
19
48
VegaLite 2.6.0
17
33
Turing 0.23.3
24
46
I haven’t seen any discussion about any precompilation regressions, is this expected for the beta?
To put it simple, to make this possible: Julia v1.9.0-beta2 is fast. Julia now compiles and stores more code during precompilation (a one-off) to save time at runtime for each single session.
(In case anyone was wondering why there are different number of packages in the environments of v1.8 and v1.10, it’s because I had to re-resolve the environment to be able to fully precompile the environment, I was getting an error without re-resolving the environment, depsite the fact DelimitedFiles isn’t anywhere to be found in the Manifest.)
I have 8. As far as I understand, PR #47797 uses half of the total cores for parallel image generation, so on a two-core machines that probably doesn’t help at all.
Going from Julia 1.8 to Julia 1.9.0-beta3, precompiling Pluto.jl (during Pkg.precompile()) went from 15 seconds to 60 seconds, pretty significant . We ran these tests many times, on different systems.
After precompilation, TTFX did improve a little, it went from 30 seconds to 20 seconds. () This is fantastic, but it certainly comes with a trade-off.
My main concern is the UX of Pkg.precompile for new Julia/Pluto users. On an older laptop, folks might be waiting 2-3 minutes on precompilation of a single package, and decide that it must be stuck or broken.
Yes, UX can do a lot to to alleviate waiting.
(Though this has improved a lot over versions of Pkg).
Related, I suspect it might make Julia friendlier if it could tell when it was compiling sth during REPL usage (instead of silently hanging for a while)
Power users, used to JIT, can then turn off this feedback if they like
Strange idea ? … or maybe a good one. I think that may make JIT more palatable to a segment of users at least initially. Actually even for experienced users you could maybe toggle it on to get some information quickly.
I suspect it might do a lot for user retention.
Even on Julia 1.9, using Plots; display(scatter(rand(1_000))) still hangs for 7 seconds, much longer than it does in Python/R/Matlab.
(Source for timing: @nilshg’s great measurements here)
Having a spinner during those seven seconds, as stupid as it sounds, will make the wait more palatable indeed. And someone just trying out Julia will be less likely to immediately drop it again. (I have no hard data of course)
Yes thats my intuition as well. Of course it’s one thing to say that’s great, and another to code it. I don’t have time at the moment. But it’s good to air the idea in any case.
Is there any way to estimate [in a manner with palatably low error, maybe ±50%] the progress of compilation time left? Could be nice to have some idea whether I have to wait 7 seconds or 70—should I twiddle my thumbs or get a coffee.
That’s a pretty large increase in TTFNotebook if measured from the installation/update of Pluto!
I haven’t tried 1.9 yet and not sure how is the typical situation with other packages - how TTFX changes 1.8 → 1.9 if precompilation is included?
Seems like a major degradation for notebooks experience in general. In Pluto, each notebook you create is a separate environment, and package versions are likely to be slightly different between them. So, running a notebook from scratch involves (at least partial) precompilation.
After reading @fons’ post above, I was terrified. All my teaching is done using Pluto notebooks, and the entire enterprise would be killed if notebooks took more than one or two minutes to precompile. Students usually are a bit impatient when waiting for the notebook to run.
I tested the latest version of Pluto v.0.19.20 (which came out yesterday) with Julia v.1.8.5 and v.1.9.0-beta3.
My system is as follows:
OS: Windows (x86_64-w64-mingw32)
CPU: 8 × Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-13.0.1 (ORCJIT, skylake)
Threads: 4 on 8 virtual cores
Version 1.9.0-beta3 is 3x faster than v.1.8.5 (≈11s vs ≈33s).
Running the notebook:
Version 1.9.0-beta3 is ≈23% faster than v.1.8.5 (≈54s vs ≈70s).
This increase in FTTX is terrific. So, either I am doing something wrong in my trails, or @fons have played a fantastic trick in the latest version of Pluto.
. We ran these tests many times, on different systems.
) This is fantastic, but it certainly comes with a trade-off.