I don’t know all the causes, but there are a few broad categories. First, it’s worth reviewing what precompilation is caching, given your observation that Makie hasn’t changed. Despite the phrase “package precompilation”, the module is not actually instantiated once per package or version. For a minimal example, a package A@1.0.0 depends on packages B@1.x.x and C@1.0.0. If I add A@1.0.0 to a fresh environment and load it, I instantiate A@1.0.0, the latest B@1.2.3, and C@1.0.0. If I add B@1.0.0 and A@1.0.0 to another fresh environment and load A, I instantiate A@1.0.0, B@1.0.0, and C@1.0.0. Obviously, both environments can use the same C@1.0.0 instance, and the different B versions need different ones. Less obviously, there are 2 A@1.0.0 instances because of the dependency on different B versions; something as simple as a B function being automatically inlined into an A function requires this. The number of environments to which you have ever added a package is a more reasonable estimate than the number of versions, but precompilation also depends on the Julia process’ compiler settings.
With that in mind, we can recognize some potential problems:
- It’s not great to accumulate hundreds of caches over time or even in one sitting of reproducing many old projects. The Julia installation thus has a cutoff for the number of instances to cache at once per package. If you cycle through enough environments across different sessions, then you’d discard a cached instance before you could reuse it. The cutoff is documented to default to 10, but I don’t know how to verify it at runtime, and as others have pointed out, it appears to be a lot less.
- Activating multiple environments within a single session can cause version conflicts with loaded package instances. Usually, we’d get a warning in the REPL to restart the session if we don’t want to re-precompile packages for a mixed and possibly broken state of environments.
- A mixed state also occurs when stacking environments. There are not many ways to stumble into that and the Pkg API isn’t designed to encourage it, but it does show up often because all of our custom environments are stacked on the Julia version’s default environment.
For a minimal example of a mixed state from the packages you mentioned, let’s say I have Revise@3.14 in the default environment and switch to another environment with only the latest GLMakie installed (both environments previously precompiled their lone packages in separate sessions):
(@v1.12) pkg> st
Status `C:\Users\Benny\.julia\environments\v1.12\Project.toml`
⌃ [295af30f] Revise v3.14.5
Info Packages marked with ⌃ have new versions available and may be upgradable.
(@v1.12) pkg> activate onlymakie\\
Activating project at `C:\Users\Benny\onlymakie`
(onlymakie) pkg> st
Status `C:\Users\Benny\onlymakie\Project.toml`
[e9467ef8] GLMakie v0.13.13
julia> using GLMakie
julia> LOAD_PATH
3-element Vector{String}:
"@"
"@v#.#"
"@stdlib"
Cool, I reached the precompile cache just fine. But GLMakie wasn’t the only package I could load. LOAD_PATH shows the stacked environments in order of where packages are searched: @ means the currently active one, @v#.# refers to the default versioned environment, and @stdlib refers to the standard library. So what happens if I load Revise from the default environment?
julia> using Revise
[ Info: Precompiling Revise [295af30f-e4ad-537b-8983-00126c2a3abe](cache misses: wrong dep version loaded (1))
Precompiling Revise finished.
1 dependency successfully precompiled in 14 seconds
Earlier I said I had already precompiled Revise for the default environment. However, GLMakie had already loaded OrderedCollections@2.0.1 here, which is not the same version (⌅ 1.8.2) in the pre-existing Revise instance. That triggers a precompile, and with JULIA_DEBUG=loading, we’d see a few relevant “Rejecting cache file” lines repeated somewhere in the wave of “Loading object cache file” lines. That wasn’t a long wait though, the other way around is much worse. In a fresh session:
julia> using Revise # we load the default instance, maybe in startup.jl
(@v1.12) pkg> activate onlymakie\\
Activating project at `C:\Users\Benny\onlymakie`
julia> using GLMakie # the wrong OrderedCollections is loaded, so...
[ Info: Precompiling GLMakie [e9467ef8-e4e7-5192-8a1a-b1aee30e663a](cache misses: wrong dep version loaded (1))
Precompiling GLMakie finished.
20 dependencies successfully precompiled in 401 seconds. 280 already precompiled.
2 dependencies precompiled but different versions are currently loaded (OrderedCollections and Preferences). Restart julia to access the new versions. Otherwise, 153 dependents of these packages may trigger further precompilation to work with the unexpected versions.
I didn’t have to precompile everything from scratch, but the <10% of dependencies took >50% the time of a full precompile. We also get the warning to restart Julia to use this precompile, though I can’t verify whether the precompile is for the mixed state or repeated for the proper state. Even if I avoid loading Revise in a new session, I have to precompile this bit for the proper GLMakie again:
(@v1.12) pkg> activate onlymakie\\
Activating project at `C:\Users\Benny\onlymakie`
julia> using GLMakie
[ Info: Precompiling GLMakie [e9467ef8-e4e7-5192-8a1a-b1aee30e663a]
Precompiling GLMakie finished.
20 dependencies successfully precompiled in 603 seconds. 280 already precompiled.
As you might have inferred from my deliberate use of an older Revise version, this wouldn’t be a problem if I used the latest Revise:
(@v1.12) pkg> st
Status `C:\Users\Benny\.julia\environments\v1.12\Project.toml`
[295af30f] Revise v3.17.0
julia> using Revise
(@v1.12) pkg> activate onlymakie\\
Activating project at `C:\Users\Benny\onlymakie`
julia> using GLMakie
This latest-update convenience is a consequence of package developers keeping up with the popular parts of the ecosystem. GLMakie also has a relatively large number of dependencies to run into loaded version conflicts, and we can often get away with smaller environments that don’t overlap in one session. In the general case, we do have to be careful about keeping incompatible environments away from each other and resyncing environments (resolve) after changes to the dev-ed packages’ project files, even as Julia improves how we can handle conflicts.
