# A visual log of loading and precompilation times for many packages over a variety of julia versions

**URL:** <https://discourse.julialang.org/t/a-visual-log-of-loading-and-precompilation-times-for-many-packages-over-a-variety-of-julia-versions/133887>\
**Category:** Profiling\
**Tags:** performance, ttfx\
**Created:** [November 14, 2025, 7:27pm UTC](https://discourse.julialang.org/t/a-visual-log-of-loading-and-precompilation-times-for-many-packages-over-a-variety-of-julia-versions/133887 "2025-11-14T19:27:47Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Krastanov](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/krastanov/32/6817_2.png) [@Krastanov](https://discourse.julialang.org/u/Krastanov)\
**Post date:** [November 14, 2025, 7:27pm UTC](https://discourse.julialang.org/t/a-visual-log-of-loading-and-precompilation-times-for-many-packages-over-a-variety-of-julia-versions/133887/1 "2025-11-14T19:27:47Z")

</div>

At [Julia Ecosystem Benchmarks Explorer](https://juliaecosystembenchmarks.github.io/web-explorer/#packages=BaseDirs&tasks=Project-Path) you can find detailed benchmarks of loading time for many packages over many different versions of julia, all interactively explorable.

This is the culmination of:

- @tecosaur setting up a [crowdsourced repository of representative TTFX benchmarks](https://github.com/tecosaur/Julia-TTFX-Snippets)
- the establishment of a nightly benchmark run over the entries of the repository
- and a quick vibecoding visualization session after discussions in [this post](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/)

It will be updated daily. Here are a few examples:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/f/c/fc0ddc78e12d15006aa6b2eebd53cce4f8156a01.png)

 ![image](https://global.discourse-cdn.com/julialang/original/3X/8/1/8195ffb5e3d12280415349d551777583a4299489.png)

I will not have the bandwidth to make improvements to it in the near future, but do not hesitate to submit patches.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [November 14, 2025, 7:38pm UTC](https://discourse.julialang.org/t/a-visual-log-of-loading-and-precompilation-times-for-many-packages-over-a-variety-of-julia-versions/133887/2 "2025-11-14T19:38:25Z")

</div>

Very cool!

I have a few questions:

- Is it identical package versions for all the dates?
- Did you not have trouble with [More graceful behavior when bisecting julia · Issue #70 · JuliaLang/PrecompileTools.jl · GitHub](https://github.com/JuliaLang/PrecompileTools.jl/issues/70) when you did the version sweep?
- How did you generate all the data in practice? Just built julia normally and ran the workload? Must have taken quite a bit of time?
- Do you have any idea what happened around this time (This is CairoMakie precompilation time which went from 185 to 300 seconds)  
 ![image](https://global.discourse-cdn.com/julialang/original/3X/1/5/153ecde885697e9b9af0d14909571384d5ffcecc.png)

---

<div class="post-metadata">

**Author:** ![Krastanov](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/krastanov/32/6817_2.png) [@Krastanov](https://discourse.julialang.org/u/Krastanov)\
**Post date:** [November 14, 2025, 7:44pm UTC](https://discourse.julialang.org/t/a-visual-log-of-loading-and-precompilation-times-for-many-packages-over-a-variety-of-julia-versions/133887/3 "2025-11-14T19:44:38Z")

</div>

All good points, I should add this to the docs

> [@kristoffer.carlsson](#):
>
> - Is it identical package versions for all the dates?

No. For each historic date I check out the General registry to its state on that date, and set it as the source of truth in the active julia depot. So all version resolutions happen as if it was that date. I manually put bounds on which date corresponds to which julia version, so a julia version is never tested on a General registry state that did not exist during the julia version lifetime.

> [@](#):
>
> - Did you not have trouble with [More graceful behavior when bisecting julia · Issue #70 · JuliaLang/PrecompileTools.jl · GitHub](https://github.com/JuliaLang/PrecompileTools.jl/issues/70) when you did the version sweep?

I use juliaup so I never manually bisected or compiled julia. There are many failures for many packages throughout this dataset, but they are just not visualized. I did not investigate why exactly the failures happen.

> [@](#):
>
> - How did you generate all the data in practice? Just built julia normally and ran the workload? Must have taken quite a bit of time?

All the benchmarks run on a lab server in the back of my office. The historical data (about a 1000 General registry snapshots) took a week or two. Now every night another benchmark run is executed (for lts, nightly, alpha, and release, as defined by juliaup).

That particular server is used for other tasks as well, most of them not computationally intensive. But the benchmarks run at high process priority with reserved resources at night, so hopefully the noise is not too bad.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [November 14, 2025, 7:45pm UTC](https://discourse.julialang.org/t/a-visual-log-of-loading-and-precompilation-times-for-many-packages-over-a-variety-of-julia-versions/133887/4 "2025-11-14T19:45:45Z")

</div>

Aha, I thought the x-axis was a sweep of commits on the julia master branch, not checkout dates of the registry. Ok.

---

<div class="post-metadata">

**Author:** ![Krastanov](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/krastanov/32/6817_2.png) [@Krastanov](https://discourse.julialang.org/u/Krastanov)\
**Post date:** [November 14, 2025, 8:00pm UTC](https://discourse.julialang.org/t/a-visual-log-of-loading-and-precompilation-times-for-many-packages-over-a-variety-of-julia-versions/133887/5 "2025-11-14T20:00:22Z")

</div>

Thankfully for any future entries under the “nightly” label, they will be both. It sounded too difficult to make that happen for historic entries too.

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [November 14, 2025, 8:12pm UTC](https://discourse.julialang.org/t/a-visual-log-of-loading-and-precompilation-times-for-many-packages-over-a-variety-of-julia-versions/133887/6 "2025-11-14T20:12:50Z")

</div>

Interesting question how such data can be interpreted when packages get updated over time. You’d assume that new features might add loading time while authors should also try to battle TTFX. The daily `nightly` measurement that’s starting now should be useful to track behavior of Julia over time. But in the historic data it seems difficult to make assessments. I’ll have to play around a little.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [November 14, 2025, 8:17pm UTC](https://discourse.julialang.org/t/a-visual-log-of-loading-and-precompilation-times-for-many-packages-over-a-variety-of-julia-versions/133887/7 "2025-11-14T20:17:33Z")

</div>

> [@jules](#):
>
> Interesting question how such data can be interpreted when packages get updated over time.

it tracks the end-user experience – “if you used CairoMakie 1 year ago and thought it was fast, it’s slower now”

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [November 14, 2025, 9:16pm UTC](https://discourse.julialang.org/t/a-visual-log-of-loading-and-precompilation-times-for-many-packages-over-a-variety-of-julia-versions/133887/8 "2025-11-14T21:16:46Z")

</div>

Sure but I’d want to know whose fault it is 😄

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [November 15, 2025, 7:01am UTC](https://discourse.julialang.org/t/a-visual-log-of-loading-and-precompilation-times-for-many-packages-over-a-variety-of-julia-versions/133887/9 "2025-11-15T07:01:02Z")

</div>

Here’s an alternative visualization. I thought that it might make sense to normalize because the packages and workloads are so different. So I took the median of a given metric within each package/workload/julia group, then rescaled those medians within each package group by dividing with the maximum value. So each package has values going up to 1 with the relative proportions between the values staying intact. The thick dots are then means over all those values for a given julia version, the small dots are the values making up the means.

There’s still considerable variation and I’m not sure if another way of normalization might be better, but at least it does reduce the impact of the different orders of magnitude of the workloads. Maybe scaling by a reference version like latest release would also work.

 ![grafik](https://global.discourse-cdn.com/julialang/original/3X/1/1/1154f585c0ed0a63d8f946090d5fa175d2d5c2b9.png)

```julia
using CSV
using Chain
using CairoMakie
using AlgebraOfGraphics
using DataFrames
using DataFrameMacros
using StatsBase
using Downloads
using SwarmMakie # ]add SwarmMakie#jkrumbiegel-patch-1

##

df = @chain begin
    "https://raw.githubusercontent.com/JuliaEcosystemBenchmarks/julia-ecosystem-benchmarks/refs/heads/jeb_logs/data/Julia-TTFX-Snippets/ttfx_snippets_data.csv"
    Downloads.download
    CSV.read(DataFrame)
end

variables = ["precompile_time", "precompile_cpu", "precompile_resident", "loading_time", "task_time", "task_cpu", "task_resident"]

spec = sum(variables) do var
    stat = median
    stat_var = "$(stat)_$var"
    stat_var_norm = "$(stat_var)_normalized"
    stat_var_norm_mean = "$(stat_var_norm)_mean"
    @chain begin
        df
        @groupby :package_name :julia_version :task_name
        @combine stat_var = median({var})
        @subset :julia_version ∉ ("alpha", "release", "lts", "nightly")
        @groupby :package_name :task_name
        @transform stat_var_norm = @bycol {stat_var} ./ maximum({stat_var})
        @aside combined = @combine (@groupby _ :julia_version) mean({stat_var_norm})
        (
            data(combined) *
                (
                    mapping(:julia_version, stat_var_norm_mean => "") *
                    visual(Scatter)
                ) +
            data(_) *
                mapping(:julia_version, stat_var_norm => "normalized median") *
                visual(Beeswarm, markersize = 4, alpha = 0.5)
        ) *
            mapping(color = :julia_version => (s -> match(r"\d\.\d+", s).match) => "minor version") *
            mapping(layout = direct(var))
    end
end |> draw(; axis = (; width = 200, height = 200, xticklabelsvisible = false))

```

---

<div class="post-metadata">

**Author:** ![Krastanov](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/krastanov/32/6817_2.png) [@Krastanov](https://discourse.julialang.org/u/Krastanov)\
**Post date:** [November 22, 2025, 6:59pm UTC](https://discourse.julialang.org/t/a-visual-log-of-loading-and-precompilation-times-for-many-packages-over-a-variety-of-julia-versions/133887/10 "2025-11-22T18:59:45Z")

</div>

If I understand your description correctly, the main reason I avoided this type of normalization is that package that support only recent julia versions make the plot incorrectly look like loading times are going up.

E.g. we have a package that supports 1.8 for which the averages are (1.0, 0.1, 0.2, 0.3, 0.3) and a package that supports 1.10 for which the averages are (missing, missing, 0.9, 1.0, 1.0), falsely pulling the summary averages way up.

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [November 22, 2025, 7:06pm UTC](https://discourse.julialang.org/t/a-visual-log-of-loading-and-precompilation-times-for-many-packages-over-a-variety-of-julia-versions/133887/11 "2025-11-22T19:06:08Z")

</div>

Yeah I noticed that effect, too and this normalization is definitely a bit simplistic. Maybe zscoring would be better or something else entirely. But just taking averages let’s the large workloads dominate the rest