# Precompilation takes ages

**URL:** <https://discourse.julialang.org/t/precompilation-takes-ages/108377>\
**Category:** General Usage\
**Created:** [January 5, 2024, 9:52am UTC](https://discourse.julialang.org/t/precompilation-takes-ages/108377 "2024-01-05T09:52:16Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![lrnv](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lrnv/32/19373_2.png) [@lrnv](https://discourse.julialang.org/u/lrnv)\
**Post date:** [January 5, 2024, 9:52am UTC](https://discourse.julialang.org/t/precompilation-takes-ages/108377/1 "2024-01-05T09:52:16Z")

</div>

Hey,

I am having some kind of an issue that does not match what I see here and what others reports.

I have an environment with the following Project.toml:

```toml
[deps]
BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
Colors = "5ae59095-9a9b-59fe-a467-6f913c188581"
Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
FiniteDifferences = "26cc04aa-876d-5657-8c51-4c34ba976000"
Format = "1fa38f19-a742-5d3f-a2b9-30dd87b9d5f8"
ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210"
HypothesisTests = "09f84164-cd44-5f33-b23f-e6b0d136a0d5"
LaTeXStrings = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f"
LaTeXTabulars = "266f59ce-6e72-579c-98bb-27b39b5c037e"
LineSearches = "d3d80556-e9d4-5f37-9878-2ab0fcc64255"
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
Measurements = "eff96d63-e80a-5855-80a2-b1b0885c5ab7"
NLSolversBase = "d41bc354-129a-5804-8e4c-c37616107c6c"
Optim = "429524aa-4258-5aef-a3af-852621145aeb"
Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
PreallocationTools = "d236fae5-4411-538c-8e31-a6e3d9e00b46"
ProgressMeter = "92933f4c-e287-5a05-a399-4b506db050ca"
RDatasets = "ce6b1742-4840-55fa-b093-852dadbb1d8b"
Revise = "295af30f-e4ad-537b-8983-00126c2a3abe"
SpecialFunctions = "276daf66-3868-5448-9aa4-cd146d93841b"
StatsBase = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
StatsPlots = "f3b207a7-027a-5e70-b257-86293d7955fd"
UnicodePlots = "b8865327-cd53-5732-bb35-84acbb429228"

```

Nothing really fancy… However, precompilation takes ages :

```julia
Precompiling project...
  55 dependencies successfully precompiled in 362 seconds. 219 already precompiled.

```

Each time I change something (add a package and/or delete one, simply update packages) I have to pay this ? Or am I missing something here ?

Btw, Plots.jl is clearly the biggest issue by eye-bowling the precompilation output it accounts for at least 70%.

This is an issue since I end up paying this almost once or twice a day – good time for coffee of course, but still.

Am I doing something wrong, or is it just the way it is ?

---

<div class="post-metadata">

**Author:** ![Amval](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/amval/32/6217_2.png) [@Amval](https://discourse.julialang.org/u/Amval)\
**Post date:** [January 5, 2024, 10:26am UTC](https://discourse.julialang.org/t/precompilation-takes-ages/108377/2 "2024-01-05T10:26:49Z")

</div>

Can you provide more details? Maybe run `versioninfo()` in the Julia REPL and coypaste the results:

```julia
julia> versioninfo()
Julia Version 1.10.0
Commit 3120989f39b (2023-12-25 18:01 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin22.4.0)
  CPU: 8 × Apple M2
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, apple-m1)
  Threads: 1 on 4 virtual cores

```

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [January 5, 2024, 10:35am UTC](https://discourse.julialang.org/t/precompilation-takes-ages/108377/3 "2024-01-05T10:35:46Z")

</div>

> [@lrnv](#):
>
> Each time I change something (add a package and/or delete one, simply update packages) I have to pay this ? Or am I missing something here ?

This shouldn’t be the case. As you see in the output `219 already precompiled`, most changes should only incur an incremental precompilation cost. In my experience `Plots` is a bit of an issue here in that it has loads of dependencies so if you change something in the environment which affects one of its dependencies that will recompile `Plots` which as you say is probably the biggest driver - ~~and having both `Plots` and `StatsPlots` doesn’t help~~ (you probably don’t need that as `StatsPlots` reexports `Plots`).

EDIT scratch that last bit, `StatsPlots` would have `Plots` as a dependency anyway, so I guess whether they are both in the `Project.toml` or not doesn’t make a difference.

---

<div class="post-metadata">

**Author:** ![lrnv](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lrnv/32/19373_2.png) [@lrnv](https://discourse.julialang.org/u/lrnv)\
**Post date:** [January 5, 2024, 10:45am UTC](https://discourse.julialang.org/t/precompilation-takes-ages/108377/4 "2024-01-05T10:45:53Z")

</div>

> [@nilshg](#):
>
> This shouldn’t be the case. As you see in the output `219 already precompiled`, most changes should only incur an incremental precompilation cost. In my experience `Plots` is a bit of an issue here in that it has loads of dependencies so if you change something in the environment which affects one of its dependencies that will recompile `Plots` which as you say is probably the biggest driver

I guess you nailed the problem. It does not recompile everything each time, but it looks like every change i made (add a pakcgae, remove one, update another one, or anything related) instantly triggers _a few_ other packages precompilation, but _almost every time_ including Plots.jl, the most problematic one…

Is there a way i can tell Julia to NOT recompile Plots.jl ? Maybe by “pinning” its version and its dependencies versions somehow ?

Better yet: is there a way I can tell it to re-do as little as possible ? Say I install a given new package, the default behavior i want is to chose the version of this new package to disrupt at least as possible the current environement state and thus trigger less precompilation. Say I update a given package, well same idea: keep compat bounds tight to avoid as much as possible work. Say I remove a package, well NOTHING should be updated. I would love such a “conservative” mode.

@Amval I am on 1.9.3, I’ll give a versionInfo once my current run is finished. I thought of updating to 1.10, but for the very reason I am posting this today i did not ^^

* * *

Edit: done abit of reading around, and found out this thread

> <https://github.com/JuliaLang/Pkg.jl/issues/3369>
>
> Running \`Pkg.add\` should not automatically update the registry. Optionally add a… warning when running \`Pkg.add\` if the registry is more than, say, 3 months old and Pkg is not in offline mode.
> 
> \## Background
> \[A recent Discourse thread\](https://discourse.julialang.org/t/first-pluto-notebook-launches-are-slower-on-julia-1-9-beta-3/93429) on the increased precompilation time with Julia 1.9 led to \[some discussion on Slack\](https://julialang.slack.com/archives/C03LD7K8HLN/p1676028682517259) about what can be done to reduce the amount of precompilation.
> 
> I don't think the Slack discussion reached a consensus, but several good suggestions were brought up that deserve an issue here on Pkg. Notably, some of the proposed solutions are good ideas \_regardless\_ of whether Julia 1.9's precompilation woes will be solved elsewhere.
> 
> \## Motivation
> The motivation is twofold: First, from a theoretical standpoint, adding a package and updating are conceptually different and orthogonal operations. You should be able to add a package without updating anything. Hence, the two operation should be separated.
> 
> From a practical point of view, developers and experienced users of Julia often create environments and modify their environments many times per day, sometimes tens of times per day. With the current state where every \`Pkg.add\` operation updates automatically, these users are going to be served new releases of various packages (dependencies of dependencies) very often, each time triggering waiting time for fetching the registry and/or package precompilation. For most users, getting the very newest version of every package is completely unnecessary - certainly getting up-to-the-hour updates is not needed.
> 
> \## Possible objections
> Q: Wouldn't this mean lots of people would get stuck on months old packages?
> A: The registry would only get stale if they never choose to run \`Pkg.update\` for \_any\_ project. And if they never do so, it's probably safe to assume they are fine running an old environment. Furthermore, we should in general expect many users explicitly want to run years old environments that they choose not to upgrade or modify.

and then this PR :

> <https://github.com/JuliaLang/Pkg.jl/pull/3378>
>
> Updated:
> 
> New \`PRESERVE\_ALL\_INSTALLED\` preserve strategy that is like \`PRESERV…E\_ALL\` except it also only selects installed versions of the new packages. 
> 
> New \`PRESERVE\_TIERED\_INSTALLED\` strategy that tries \`PRESERVE\_ALL\_INSTALLED\` first.
> 
> To opt-in to \`PRESERVE\_TIERED\_INSTALLED\` being the default set the env var \`JULIA\_PKG\_PRESERVE\_TIERED\_INSTALLED=true\` 
> 
> Discussion in:
> \- https://discourse.julialang.org/t/first-pluto-notebook-launches-are-slower-on-julia-1-9-beta-3/93429/158
> \- https://github.com/JuliaLang/Pkg.jl/issues/3369

Looks like others were having this kind of issues before me 🙂 Now i just have to understand how to use these new functionalities to solve my issue…

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [January 5, 2024, 11:56am UTC](https://discourse.julialang.org/t/precompilation-takes-ages/108377/5 "2024-01-05T11:56:18Z")

</div>

> [@lrnv](#):
>
> Maybe by “pinning” its version and its dependencies versions somehow ?

[The docs on pinning a package version](https://pkgdocs.julialang.org/v1/managing-packages/#Pinning-a-package). Of course you have to keep in mind that pinning that package also restricts other packages indirectly because of dependency compatibility. `Pkg.add` also has [settings for version resolution](https://pkgdocs.julialang.org/v1/api/#Pkg.add) but I’ve nevered changed the default because the rest sound intimidating.

6 minutes does seem like a lot, over 4 of it is parts of Plots. It _is_ a hefty package so maybe this isn’t actually unusual.

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [January 5, 2024, 12:14pm UTC](https://discourse.julialang.org/t/precompilation-takes-ages/108377/6 "2024-01-05T12:14:01Z")

</div>

I also get Plots precompiled sometimes up to 10 times per day, but it usually takes less than 30 seconds for me, so 70% of 6 minutes still sounds like a lot

---

<div class="post-metadata">

**Author:** ![lrnv](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lrnv/32/19373_2.png) [@lrnv](https://discourse.julialang.org/u/lrnv)\
**Post date:** [January 5, 2024, 1:08pm UTC](https://discourse.julialang.org/t/precompilation-takes-ages/108377/7 "2024-01-05T13:08:13Z")

</div>

> [@Benny](#):
>
> [The docs on pinning a package version](https://pkgdocs.julialang.org/v1/managing-packages/#Pinning-a-package). Of course you have to keep in mind that pinning that package also restricts other packages indirectly because of dependency compatibility. `Pkg.add` also has [settings for version resolution](https://pkgdocs.julialang.org/v1/api/#Pkg.add) but I’ve nevered changed the default because the rest sound intimidating.
> 
> 6 minutes does seem like a lot, over 4 of it is parts of Plots. It _is_ a hefty package so maybe this isn’t actually unusual.

I am not sure pinning Plots.jl is enough: if one of its dependency is changed (within compat bounds) it will still precompile Plots.jl again right ?

I think what I want is `PRESERVE_TIERED_INSTALLED`. The docs says

> To change the default strategy to `PRESERVE_TIERED_INSTALLED` set the env var `JULIA_PKG_PRESERVE_TIERED_INSTALLED` to true.

Do someone know how i can setup this environement variable globally on my machine (windows machine) ?

> [@baggepinnen](#):
>
> I also get Plots precompiled sometimes up to 10 times per day, but it usually takes less than 30 seconds for me, so 70% of 6 minutes still sounds like a lot

Indeed, maybe there is a reason mine takes so long… but i do not know which one. Might be related to the frequency of re-precompilation ? or to the number of stuff in the env ? dunno.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [January 5, 2024, 1:58pm UTC](https://discourse.julialang.org/t/precompilation-takes-ages/108377/8 "2024-01-05T13:58:20Z")

</div>

Just freeze your Project.toml. Add [PkgHelpers.jl](https://github.com/ufechner7/PkgHelpers.jl) to your global environment, and then launch Julia in your project environment, and then execute:

```julia
using Pkg, PkgHelpers
freeze(Pkg)

```

The next time you add a package only that package will get compiled.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 5, 2024, 5:54pm UTC](https://discourse.julialang.org/t/precompilation-takes-ages/108377/9 "2024-01-05T17:54:23Z")

</div>

> [@ufechner7](#):
>
> Just freeze your Project.toml.

I add

```julia
import Pkg
Pkg.UPDATED_REGISTRY_THIS_SESSION[] = true

```

to my `~/.julia/config/startup.jl` so that it doesn’t update the whole registry every time I add a package (only if I explicitly run `update`).

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [January 5, 2024, 6:09pm UTC](https://discourse.julialang.org/t/precompilation-takes-ages/108377/10 "2024-01-05T18:09:53Z")

</div>

I think the only way to do this is building a sysimage with Plots in it.

EDIT that was supposed to be an answer to “pin Plots so it doesn’t get updated” but I failed at Discourse.

---

<div class="post-metadata">

**Author:** ![lrnv](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lrnv/32/19373_2.png) [@lrnv](https://discourse.julialang.org/u/lrnv)\
**Post date:** [January 5, 2024, 6:36pm UTC](https://discourse.julialang.org/t/precompilation-takes-ages/108377/11 "2024-01-05T18:36:22Z")

</div>

So I have several options there…

1. @ufechner7 suggests adding to startup.jl:

```julia
using Pkg, PkgHelpers
freeze(Pkg)

```

1. @stevengj suggests insterad adding

```julia
import Pkg
Pkg.UPDATED_REGISTRY_THIS_SESSION[] = true

```

1. Pkg’s documentation suggests adding

```julia
ENV["JULIA_PKG_PRESERVE_TIERED_INSTALLED"] = true

```

1. @nilshg suggests building a sysimage with Plots.jl in it.

I am not clear on the differents ups and downs of the four approaches…

- I dont thing option 4 is the way to go as there is not only Plots.jl but i want a more general behavior, and I do not want to freeze stuff, I want to be able to update packages _if i ask for it_ (so option 3 is probably the way to go)
- After reading PkgHelpers.jl docs it looks like `freeze()` will add stuff to compat entries of the project, so option 1 looks a bit invasive to my taste…
- I am not understanding correctly what option 2 really does. Maybe 2+3 is the right call for me ?

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [January 5, 2024, 6:42pm UTC](https://discourse.julialang.org/t/precompilation-takes-ages/108377/12 "2024-01-05T18:42:39Z")

</div>

As I understand it 2 basically tells Julia not to update the general registry. This is often the source of “unnecessary” precompilation in the sense that some dependency of a dependency of a dependency of Plots releases some patch release, causing recompilation of the entire Plots stack.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [January 5, 2024, 6:51pm UTC](https://discourse.julialang.org/t/precompilation-takes-ages/108377/13 "2024-01-05T18:51:29Z")

</div>

> [@lrnv](#):
>
> @ufechner7 suggests adding to startup.jl:

Well, I never suggested to add this to startup.jl. Just execute this once.

---

<div class="post-metadata">

**Author:** ![lrnv](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lrnv/32/19373_2.png) [@lrnv](https://discourse.julialang.org/u/lrnv)\
**Post date:** [January 5, 2024, 6:53pm UTC](https://discourse.julialang.org/t/precompilation-takes-ages/108377/14 "2024-01-05T18:53:24Z")

</div>

> [@ufechner7](#):
>
> Well, I never suggested to add this to startup.jl. Just execute this once.

yes i extrapolated sorry

---

<div class="post-metadata">

**Author:** ![ianshmean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ianshmean/32/216042_2.png) [@ianshmean](https://discourse.julialang.org/u/ianshmean)\
**Post date:** [January 5, 2024, 7:04pm UTC](https://discourse.julialang.org/t/precompilation-takes-ages/108377/15 "2024-01-05T19:04:55Z")

</div>

I’d recommend just doing 3 in your `.julia/config/startup.jl`

```julia
ENV["JULIA_PKG_PRESERVE_TIERED_INSTALLED"] = true

```

I’ve been running with that since 1.9 came out and I haven’t seen any issues.

The reason it’s not the default is that some were concerned that it would mean users would generally get stuck on older package versions, but that hasn’t been a noticeable issue for me.