New Julia versions, higher Pluto loading times

General Usage
1

I will start teaching two courses (one undergraduate with 350 students and the other a master’s course with 40 students) at the beginning of February. None of those courses involves CS students, which implies that the computational side of the material will need to be kept at an introductory level. It is not the first time we are doing this; actually, it is the tenth semester.

In our previous experience, students showed no distaste for the Julia programming language and loved working with Pluto notebooks, which they found an amazing tool. However, they complain loudly about how long it takes to load a pretty regular Pluto notebook. In the first years of this journey, we expected that as new Julia versions were released, these large loading times would be significantly reduced or at least become more tolerable, on par with other languages (e.g., Python and JavaScript, given that I am only considering reactive notebooks here).

However, such an expectation has proved wrong so far. As new Julia versions kept coming, not only did precompilation times keep increasing, but so did running times. I checked this forum, and several issues have already been raised about similar concerns. For example, three years ago, @fonsp raised an issue about the time it takes to launch a Pluto notebook for the first time with version 1.9 here, and more recently, he raised concerns about precompilation times in general here. I am also aware of @krastanov’s recent issue here.

After reading a large number of contributions across all those threads (and there are others as well), I am left with the impression that most people consider that there is no significant problem with either precompilation or run times across the new Julia versions. I may have developed a wrong impression here, as my programming skills are mediocre, but that is how I feel from my readings.

However, as a Julia and Pluto user, I have a problem. I have several notebooks that I use in my teaching, and as time has gone by and new Julia versions have arrived, loading times have gotten worse, not better. Over these five years, the notebooks and packages I use have remained more or less the same, the students’ computers have become significantly better, but to my surprise, the loading times have increased significantly!

Let me clarify one important point. In my particular case, I am not highly concerned about precompilation time in general, because the second time I load a Pluto notebook, the precompilation times are tolerable: e.g., 4 or 5s for a set of packages that includes HypertextLiteral, PlutoUI, ColorSchemes, PlutoPlotly, SparseArrays, PlotlyKaleido, CSV, DataFrames, Dates, LinearAlgebra, NLsolve, StatsBase, CommonMark. What surprises me is the large amount of run time (evaluating cells) it takes on my computer (a good Windows 11 machine): from 16.1s in Julia 1.10.10 to 20.2s in Julia 1.12.4. If the packages are the same, the Pluto version is the same (0.20.21), why does Julia 1.12.4 take around 125% of the time that Julia 1.10.10 requires to evaluate the cells in a pretty simple notebook? But the same happens during precompilation time (second run) in the same notebook: Julia 1.12.4 takes about 135.7% longer than Julia 1.10.10.

As I am well aware of the limitations of my programming skills, I did a similar test using one of @fons Pluto’s notebooks created for a course on Bayesian Machine Learning at the TU Eindhoven. The second time his “Bayesian Machine Learning” notebook was loaded, the times are as follows:

  • Julia 1.10.10: Packages precompilation (2.3s), Running code (8.3s)
  • Julia 1.12.4: Packages precompilation (4.3s), Running code (12.2s)

This data suggests that the precompilation time in Julia 1.12.4 is more than 87% that in Julia 1.10.10 for a set of packages that includes Plots, LaTeXStrings, Plots, PlotMeasures, Distributions, StatsPlots, SpecialFunctions, Random, BmlipTeachingTools, and MarkdownLiteral. On the other hand, the running time in Julia 1.12.4 is 47% higher than in version 1.10.10.

Please do not misunderstand me: I am not criticizing anyone or anything, and I must be thankful to all those who put time and effort into developing open-source tools that I use for free. I have been using Julia since 2019, and I love the language. However, the trend I have been witnessing in precompilation and runtime hurts how students (and my colleagues) view the Julia programming language and the Pluto notebook. As people become aware that a reactive notebook in Python is currently around four times faster than a Pluto notebook, we who have pushed for the use of such a beautiful language and tools in our teaching duties will be in trouble sooner or later.

10 Likes
2

Well, why not just stick to Julia 1.10 for your teaching?

The other question is, why is the runtime higher? This should not be the case, but I do not know how to measure it correctly for Pluto notebooks. So perhaps someone else can chime in.

3

As far as I understand, precompilation times are getting higher because packages are precompiling more code in an attempt to reduce the time to first run. The downside is that everyone is paying the cost of compiling code that they will not use. I don’t know an easy solution to that.

Running times shouldn’t get worst, though. Are those “first” running times only? If you demand all cells to be reevaluated, is there a noticeable performance difference?

(Concerning specifically Pluto, I can only say that my attempts to use it always were blocked by this kind of issue - in my teaching experience I always ended using plain coding, but I’m sure Pluto can help a lot)

1 Like
4

And not just for teaching… I tried 1.11 and 1.12 when they came out, but because of exactly this reason, I’m staying on 1.10. That version struck a great balance, and has by far the shortest compilation times among recent Julias – as your examples show as well, @VivMendes .
This trend of slower and slower TTFX doesn’t seem to be reversing yet, see Package precompile time regressions (nightly worse than 1.12 worse than 1.11 worse than 1.10) · Issue #58246 · JuliaLang/julia · GitHub.

Not sure whether that’s the case… The same package versions take much more time on eg 1.12 compared to 1.10.

1 Like
5

1.10 has some known compiler bugs that are fixed in 1.11 and 1.12.

6

But are they relevant for teaching?

7

@VivMendes for what it’s worth, I think your are much less alone in your view than you think. Indeed, your view of the state of affairs is probably typical:

  • Latency has been ranked as the #1 technical problem of Julia in every single year’s Julia User and Developer Survey that I have seen
  • Latency has been a favoured topic by the core devs at previous years’ JuliaCon.
  • There has been several statements by some of the core devs (I recall Jeff Bezanson, specifically, saying this) that low latency was their number one priority ‘feature’ when developing Julia.
  • Perhaps beside lack of intefaces, the latency regression since Julia 1.10 may be the top complaint on this Discourse the last few years.

I think where the disconnect happens is: If everyone agrees that latency fucking sucks, and the devs worked so hard to lower it throughout the years, why hasn’t there been put more effort in preventing the regression, or handling it when it first appeared?

Which I can’t really answer, unfortunately. Purely speculating, I would guess it’s a combination of:

  • There isn’t really any pool of money specifically set aside to hire people to reduce latency, since it kind of hits everyone and a 30% latency reduction won’t make a decisive advantage for any product.
  • Julia lacks the foundational design and features to keep latency low - any progress is just going to regress again.
  • There has been some work on large scale changes that may improve this, such as better caching, but these are deep, thorough changes that has taken longer than anticipated.
6 Likes
8

Yes, it is unfortunate to see that this tendency shows no sign to revert. On the contrary. Compile times on nightly jumped again.

julia> VERSION
v"1.10.10"

julia> using GMT
Precompiling GMT finished.
  1 dependency successfully precompiled in 53 seconds. 80 already precompiled.

julia> VERSION
v"1.12.4"

julia> using GMT
Precompiling GMT finished.
  1 dependency successfully precompiled in 71 seconds. 106 already precompiled.

julia> VERSION
v"1.13.0-beta1"

julia> using GMT
[ Info: Precompiling GMT [5752ebe1-31b9-557e-87aa-f909b540aa54]
Precompiling GMT finished.
  1 dependency successfully precompiled in 76 seconds. 105 already precompiled.

julia> VERSION
v"1.14.0-DEV.1549"

julia> using GMT
[ Info: Precompiling GMT [5752ebe1-31b9-557e-87aa-f909b540aa54]
Precompiling GMT finished.
  1 dependency successfully precompiled in 100 seconds. 105 already precompiled.

And when seeing the time it takes to run the tests, the situation is much worse (but that goes back to version 1.11).

Captura de ecrã 2026-01-21 182621
Captura de ecrã 2026-01-21 1826211320×517 128 KB

1 Like
9

How can one perform a reliable test of that? Meaning, with the guarantee that the same packages and versions are involved in each test?

In your example not even the number of packages being precompiled seem the same.

(the aim would be to isolate the effect of the Julia versions vs. the dependency tree)

10

I don’t know why the number of already precompiled is different across versions, but no other package compilation was included in the reported times. Just deleted the cache of nightly version, recompiled and got 96 sec this time but I normally I get > 100 s.

Other than stdlibs and *_jll artifacts, GMT.jl only depends on Tables.jl and PrecompileTools.jl

1 Like
11

That is one possibility; unfortunately, it is not a good one. The point is that, with these loading times, a regular Pluto notebook can take a very long time to load on students’ computers … even if we stick with v1.10, and they hate it (can we imagine how nervous students are when they receive a notebook with a test and it takes a long, long time to load?)

If we use python marimo notebooks, the same exact material takes 4 or 5 times less time to load. Until 18 months ago, there was no serious problem because there was no sound alternative to Julia and Pluto that met our needs: there was no reactive notebook in Python, and using Observable and JavaScript did not look like a good option. Now, there is marimo, and we can ignore it at our peril. Well, my colleagues are not ignoring it.

There is simply something strange in Julia’s evolution over the last 5 years (at least as far as Pluto is concerned): with better computers, students take longer to load a pretty normal notebook than they did 5 years ago. In my humble opinion, it should be the other way around.

Yes, they are “first” runs when a notebook is loaded, because that is what matters when we load a reactive notebook. If cells are reevaluated, there is no problem; we are back to speedy Julia.

I can assure you that my notebooks are basically the same over this 5-year period, and the packages are the same as well (with one minor exception or another). And I do not believe that Pluto development has anything to do with that latency we have been considering. Therefore, the increase in loading times appears to be a problem with the Julia development itself.

1 Like
12

My suspicion is that currently nobody really has a concrete idea why recent Julia versions are slower in many aspects than previous ones. At least, if anybody knows, they have not posted it anywhere I could see it.

We often see timings that show ttfx and loading latency, but those cannot tell us what computation actually takes longer and why. Maybe it’s a lot of small different bits that together amount to the worse experience we’re getting currently. For example, the Memory change was said to pull some C functionality into Julia, offering the potential for the compiler to do more advanced optimizations. But those might just take more time than previous passes.

It’s all conjecture though without actual measurements. I’d be interested in that but don’t know the compiler internals enough to know where it would make sense to look.

1 Like
13

I wonder how much the libs excision has to do with this. They started after 1.10, when all these regressions also started.

14

Can you clarify what you’re calling precompilation here and how you’re getting these numbers? Is it the packages’ workloads when installed, or the compilation of the code in the notebook itself?