# Julia 1.9.0-beta2 precompilation taking almost twice as long

**URL:** <https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498>\
**Category:** General Usage\
**Tags:** question, precompilation\
**Created:** [January 4, 2023, 11:27am UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498 "2023-01-04T11:27:08Z")\
**Posts on this page:** 19\
**Page:** 2

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [January 22, 2023, 3:17pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/21 "2023-01-22T15:17:39Z")

</div>

> [@aplavin](#):
>
> That’s a pretty large increase in TTFNotebook if measured from the installation/update of Pluto!

To me, it feels worth optimising for latency after precompilation because the number of notebooks started for a given Pluto version is presumably a lot larger than the number of times Pluto is updated.

Using some of the numbers above, you only need to start 4 notebooks to have gained time from the new system.

---

<div class="post-metadata">

**Author:** ![jdad](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jdad/32/4739_2.png) [@jdad](https://discourse.julialang.org/u/jdad)\
**Post date:** [January 22, 2023, 3:34pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/22 "2023-01-22T15:34:28Z")

</div>

May be he meant `delete the .julia/compiled/v1.9` folder, which is an operation I do in order to clean the compilation cache. I agree that deleting something in .julia could be dangerous - then, is there a Julia-approved way to clean the compile cache, eg, by calling some function inside the REPL?

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [January 22, 2023, 3:42pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/23 "2023-01-22T15:42:18Z")

</div>

> [@kristoffer.carlsson](#):
>
> To me, it feels worth optimising for latency after precompilation because the number of notebooks started for a given Pluto version is presumably a lot larger than the number of times Pluto is updated.

Precompilation of Pluto itself is only part of the equation. The more notebooks one has, the more precompilation time they pay.

Environments of each notebook need to be precompiled after: updating julia, updating any package in that env, adding/removing deps in those envs, cache cleanup, creating/loading new notebook, maybe something else.  
So, if precompilation of many packages actually got much slower, as this thread suggests, then TTFX visible by the majority of Pluto users would significantly suffer. Yes, running the same notebook again without changing anything would be faster, but both scenarios are very common.

---

<div class="post-metadata">

**Author:** ![VivMendes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vivmendes/32/16477_2.png) [@VivMendes](https://discourse.julialang.org/u/VivMendes)\
**Post date:** [January 22, 2023, 4:42pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/24 "2023-01-22T16:42:56Z")

</div>

@ufetcner7, no, I did not delete .julia folder. I am a user, and I am always afraid of taking some sophisticated steps into the unknown. I put myself in the footsteps of my students: I need to start a new Julia session, import and run Pluto, and open a notebook. In my trials, I did precisely this several times.

Apparently, the latest version of Pluto (v.0.19.20) does the job significantly faster in Julia 1.9.0-beta3 than in v.1.8.5 (I tested with different notebooks and different packages). In 1.9.0-beta3, importing and running Pluto is around three times faster (2x faster in another computer) than in v.1.8.5. Still, the gains are also visible when opening a notebook (23% and 15% in two different computers). I think the time it takes to open a notebook is the most crucial factor for users, like myself or my students.

Nevertheless, I kept in my notes that the first time I used Pluto (v.0.19.20) with Julia 1.9.0-beta3, it took 110s to precompile Pluto and 4’45’’ to open the major notebook I used in these trials. I understand that such a long time may be detrimental for some users, but for those that use Pluto notebooks a lot, the crucial point is how long it takes to have an average-sized notebook running on one’s computer.

---

<div class="post-metadata">

**Author:** ![t-bltg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/t-bltg/32/25526_2.png) [@t-bltg](https://discourse.julialang.org/u/t-bltg)\
**Post date:** [January 22, 2023, 5:25pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/25 "2023-01-22T17:25:56Z")

</div>

> [@jdad](#):
>
> is there a Julia-approved way to clean the compile cache, eg, by calling some function inside the REPL?

If your package is loaded, you can use `Base.compilecache(Base.module_keys[the_module])` to execute and cache precompile statements again.  
I also use `rm.(Base.find_all_in_cache_path(Base.module_keys[the_module]))` to remove the `.ji` files from within the REPL.

---

<div class="post-metadata">

**Author:** ![jdad](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jdad/32/4739_2.png) [@jdad](https://discourse.julialang.org/u/jdad)\
**Post date:** [January 22, 2023, 6:02pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/26 "2023-01-22T18:02:37Z")

</div>

Thanks!

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [January 22, 2023, 8:03pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/27 "2023-01-22T20:03:51Z")

</div>

> [@aplavin](#):
>
> The more notebooks one has, the more precompilation time they pay.

Not really, the more unique packages one has, the more you have to precompile.

> [@aplavin](#):
>
> Environments of each notebook need to be precompiled after:

> updating julia

Yes

> updating any package in that env

Only those packages that update (and their reverse dependencies)

> adding/removing deps in those envs,

Removing deps does not require precompilation, adding one is similar to updatting.

> cache cleanup

Not sure what this is.

> new notebook

Only if it uses other packages than the other notebooks.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [January 22, 2023, 8:20pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/28 "2023-01-22T20:20:00Z")

</div>

> [@kristoffer.carlsson](#):
>
> Not really, the more unique packages one has, the more you have to precompile.

Unique package-version combinations, including transitive dependencies. It’s very common to create two notebooks within a few days and end up with at least one different package version in the manifest.

> [@kristoffer.carlsson](#):
>
> > new notebook
> 
> Only if it uses other packages than the other notebooks.

Same.

> [@kristoffer.carlsson](#):
>
> > cache cleanup
> 
> Not sure what this is.

Julia doesn’t store all precompiled codes for all package-versions forever, right?..

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [January 22, 2023, 8:26pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/29 "2023-01-22T20:26:11Z")

</div>

> [@giordano](#):
>
> Please refrain from doing such dangerous suggestions again.

Well, if you want to measure the precompilation time of a package you must start with an empty .julia folder. Of course you can also rename it before doing the measurement which is more safe.

I often do my measurements in a virtual machine which I do not use for other tasks which is also safe. But deleting the .julia folder should always be safe if you have your own code in a git repository on a server.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [January 22, 2023, 8:29pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/30 "2023-01-22T20:29:08Z")

</div>

> [@ufechner7](#):
>
> Well, if you want to measure the precompilation time of a package you must start with an empty .julia folder

That’s inaccurate. You only need to delete `compiled/vX.Y` subdirectory of the depot, and that’s safe to do because that only contains autogenerated files.

> [@ufechner7](#):
>
> Of course you can also rename it before doing the measurement which is more safe.

Which you didn’t suggest. People believing that your suggestion was safe may lose their development code.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [January 22, 2023, 8:30pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/31 "2023-01-22T20:30:38Z")

</div>

> [@giordano](#):
>
> People believing that your suggestion was safe may lose their development code.

If they don’t have it in git which is stupid in the first place. But you are right, scientists are often not programmers, sometimes I forget that, sorry…

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [January 22, 2023, 8:31pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/32 "2023-01-22T20:31:59Z")

</div>

> [@ufechner7](#):
>
> If they don’t have it in git which is stupid in the first place.

People may have not had the time to push the latest local work.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [January 22, 2023, 8:36pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/33 "2023-01-22T20:36:10Z")

</div>

> [@giordano](#):
>
> You only need to delete `compiled/vX.Y` subdirectory of the depot, and that’s safe to do because that only contains autogenerated files.

Thanks a lot, I did not know that…

---

<div class="post-metadata">

**Author:** ![j-fu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/j-fu/32/11373_2.png) [@j-fu](https://discourse.julialang.org/u/j-fu)\
**Post date:** [January 22, 2023, 10:58pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/34 "2023-01-22T22:58:35Z")

</div>

Idea for @fonsp : (which is related to [Allow to provide environment at notebook start · Issue #1788 · fonsp/Pluto.jl · GitHub](https://github.com/fonsp/Pluto.jl/issues/1788) ): it would be nice to have a tool which sets the environment for a notebook. Or any other way to make a bunch of pluto notebooks using the same package versions.

---

<div class="post-metadata">

**Author:** ![Raf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raf/32/3383_2.png) [@Raf](https://discourse.julialang.org/u/Raf)\
**Post date:** [January 22, 2023, 11:48pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/35 "2023-01-22T23:48:04Z")

</div>

In my experience notebooks tend to end up with different package versions in their manifest. Just because you made them on different days. So using the same packages doesn’t really help.

---

<div class="post-metadata">

**Author:** ![fonsp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fonsp/32/222349_2.png) [@fonsp](https://discourse.julialang.org/u/fonsp)\
**Post date:** [January 23, 2023, 8:56pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/36 "2023-01-23T20:56:20Z")

</div>

Hey @VivMendes! It looks like these benchmarks were done with a hot cache. Depending on your situation, this could actually be realistic (when students will often run the same notebook), and it demonstrates the advantage of Julia 1.9.

We have been running benchmarks where we also clear the cache to simulate the experience of first-time Julia users. For a class like [https://computationalthinking.mit.edu/](https://computationalthinking.mit.edu/) , which has lots of notebooks with different packages, and (hopefully) lots of people trying to run various lectures and homeworks on their own computer or binder, the increased precompilation times are more noticeable.

I wrote more about my benchmarks and experience in this post 👉 [Julia 1.9 beta3 is much slower for first-time users](https://discourse.julialang.org/t/julia-1-9-beta3-is-much-slower-for-first-time-users/93429)

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [January 23, 2023, 9:17pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/37 "2023-01-23T21:17:33Z")

</div>

I wonder if the solution to slow precompilation in Julia will be similar to what the Python community did with conda, i.e. a set of environments precompiled and stored in the cloud with a set of common popular packages.

If packages such as Plots.jl, Makie.jl and other heavy packages that are widely used lived in a central hub with precompiled sysimages, then end-users could ask the package manager to download these generic binaries from the central hub instead of compiling locally? This would improve the user experience for most users who do not want to modify packages, but just use stable releases.

If the no-free-lunch theorem applies here, TTFX moves to precompilation, and we will never be satisfied with the total time. We could exploit good internet connection to alleviate the problem.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [January 23, 2023, 9:30pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/38 "2023-01-23T21:30:59Z")

</div>

Would it be possible to store precompiled package images in the cloud? That could be more fine-grained than system images…

I mean, after [Make precompile files relocatable/servable · Issue #47943 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/47943) has been implemented…

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [January 30, 2023, 5:19pm UTC](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498/39 "2023-01-30T17:19:46Z")

</div>

> [@juliohm](#):
>
> I wonder if the solution to slow precompilation in Julia will be similar to what the Python community did with conda, i.e. a set of environments precompiled and stored in the cloud with a set of common popular packages.

I’ve been considering this. You would precompute some “standard slices” of the current set of registered packages that cover the entire registry when you union them together, but which have some incompatibility between them. You can publish these along with the latest registry. Instead of resolving on the client side, you just find a slice that has all the packages you want; if your needs can’t be satisfied by any precomputed slice, then there’s some inherent incompatibility in what you’re trying to resolve. Benefits: fast, simple, very predictable resolution of requirements and you can precompile entire slices and serve the binaries. Drawbacks: I’m not sure about the feasibility of computing these, but I have some ideas.

[Previous page](https://discourse.julialang.org/t/julia-1-9-0-beta2-precompilation-taking-almost-twice-as-long/92498.md?page=1)