# Startup time of 1000 packages – 53% slower in Julia 1.12 vs 1.10

**URL:** <https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343>\
**Category:** Internals & Design\
**Tags:** performance, ttfx, latency\
**Created:** [April 23, 2025, 7:36pm UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343 "2025-04-23T19:36:30Z")\
**Posts on this page:** 20\
**Page:** 3

<div class="post-metadata">

**Author:** ![ForceBru](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/forcebru/32/21389_2.png) [@ForceBru](https://discourse.julialang.org/u/ForceBru)\
**Post date:** [April 28, 2025, 12:16pm UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/41 "2025-04-28T12:16:30Z")

</div>

> [@tecosaur](#):
>
> separate family of multi-package snippets if there’s demand/value

Perhaps each snippet could focus on a particular _task_, like:

- computing a gradient (I have a simple piece of code which shows that time-to-first-gradient with ForwardDiff is _60%_ worse in 1.12 compared to 1.11);
- plotting a simple function like `Plots.plot(range(-5,5,100), sin)`;
- minimizing a simple function using plain-Julia minimization algorithms like those in Optim.jl;
- solving a system of linear equations using plain-Julia algorithms in 3rd-party packages like LinearSolve.jl;
- fitting a simple normal distribution using MCMC with Turing.jl.

The goal with “plain-Julia” is to avoid calling into precompiled C/C++/FORTRAN code.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [April 28, 2025, 12:17pm UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/42 "2025-04-28T12:17:57Z")

</div>

> [@gdalle](#):
>
> That may not be possible for some of the most used packages, which require a combination of dependencies to do anything at all.

Hmmm, that’s a good point. I was just hoping that in the spirit of keeping things as simple as they can be that we’d be able to get by with something like:

```julia
# --- snippet --
# julia: 1.7
# mypkg: 0.3
# author: @name

MyPkg.dothing()

# --- snippet ---
# julia: 1.9
# mypkg: 0.4
# author: @someone

MyPkg.anotherthing()

# ...

```

this would make it possible to simply to have a single `M/MyPkg.jl` file for each package.

I guess we could do `M/MyPkg/<task>/{Project.toml,task.jl}` instead.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [April 28, 2025, 12:25pm UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/43 "2025-04-28T12:25:56Z")

</div>

> [@tecosaur](#):
>
> I was just hoping that in the spirit of keeping things as simple as they can be that we’d be able to get by with something like:

I appreciate the concept but it seems to me that `Project.toml` is already capable ot storing package names and versions, julia versions and authors. From there, it seems almost more complicated to invent a new format?

---

<div class="post-metadata">

**Author:** ![Datseris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/datseris/32/13406_2.png) [@Datseris](https://discourse.julialang.org/u/Datseris)\
**Post date:** [April 28, 2025, 12:36pm UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/44 "2025-04-28T12:36:40Z")

</div>

> [@ForceBru](#):
>
> Perhaps each snippet could focus on a particular _task_, like:

+1, exactly. For most packages for which people care about their TTFX they already load so many dependencies upon `using` I am not sure if there is a practical reason to focus on the _package_ instead of the _task_, provided the task is central to a package and is done with that package.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [April 28, 2025, 1:14pm UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/45 "2025-04-28T13:14:16Z")

</div>

> [@gdalle](#):
>
> From there, it seems almost more complicated to invent a new format?

Pah, that’s not a format, it’s just a few special comments 😛

That said, I’m taking from this discussion that orienting around tasks with a “main package” (not necessarily the only package) is a better approach.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [April 28, 2025, 11:58pm UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/46 "2025-04-28T23:58:38Z")

</div>

> [@tecosaur](#):
>
> Ok, seems there’s interest. I’ll spin something up.

Update: I’ve got something coming, I’ll probably have something to share in the next few days.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [April 30, 2025, 6:49am UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/47 "2025-04-30T06:49:49Z")

</div>

> [@tecosaur](#):
>
> 1. Given the different ways you might want to resolve package/dep versions, I feel like adding full manifest information is a bit much. I’m leaning towards just a minimum Julia + package version

I think a Manifest is pretty much required here if you want this information to be useful.

New versions of packages can do things like add precompilation workflows, or add large amounts of code which could substantially change latency / ttfx in a non-breaking way.

If the dependencies of a package are not fixed, you risk a huge amount of noise / systematic error being folded into your measurements, and you make your conclusions less reproducible.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [April 30, 2025, 2:09pm UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/48 "2025-04-30T14:09:16Z")

</div>

> [@Mason](#):
>
> I think a Manifest is pretty much required here if you want this information to be useful.

Depends on what “this information” is. With GitHub CI being on shared machines, I don’t think it’s feasible to get good quality benchmark results from it.

So, this set of trial workloads will be have to be run by people wanting to do benchmarks. Depending on what you’re trying to measure though, different resolution strategies make sense:

- If you want to compare what people experienced at the time points of each Julia version, you’d want to resolve with registries of the same year as each Julia release.
- If you want to compare Julia itself, then you might want to try to use the same code to the greatest extent possible, and construct the manifest from the lowest supported Julia version and just re-resolve (not upgrade) with newer versions.
- When trying to compare Julia versions, you might not care about Julia versions before a certain point (say 1.6), in which case you’d want to initially resolve with version `max(1.6, min-ver)`.

So, I don’t think there’s a single clear answer to what the Manifest _should be_, particularly if the repo itself isn’t benchmarking.

Despite that, simply having a set of tasks + minimum Julia versions like this is still hugely valuable as it allows us to start looking at these questions.

---

<div class="post-metadata">

**Author:** ![haakon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/haakon/32/42692_2.png) [@haakon](https://discourse.julialang.org/u/haakon)\
**Post date:** [April 30, 2025, 7:00pm UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/49 "2025-04-30T19:00:24Z")

</div>

> [@Datseris](#):
>
> First of all my impression is that download speed is practically instant. I’ve never noticed any meaningful amount of time to download.

Download speed can vary dramatically depending on a [user’s internet service](https://worldpopulationreview.com/country-rankings/internet-speeds-by-country). In my experience this can completely dwarf precompile/load times on some internet connections.

I would find it useful if the code/notebook calculated the size of these packages (including their full dependency tree). A potentially reasonably simple knob to add to the notebook is a few options for what your internet speed is [e.g. 1Mbit/s, 10Mbit/s, 100Mbit/s, 1000Mbit/s]. Then the plot could calculate `[package (+deps) size] / [download speed]` and use that as its `Installation time` measure.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [May 3, 2025, 9:08am UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/50 "2025-05-03T09:08:14Z")

</div>

> [@tecosaur](#):
>
> Update: I’ve got something coming, I’ll probably have something to share in the next few days.

> **[GitHub - tecosaur/Julia-TTFX-Snippets: A collection of TTFX workloads for Julia packages,...](https://github.com/tecosaur/Julia-TTFX-Snippets)**
>
> A collection of TTFX workloads for Julia packages, for longitudinal performance testing.

It’s not everything I want it to be, but it’s a good starting point I hope!

Do check it out and let me know what you think 😃

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [May 4, 2025, 6:29am UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/51 "2025-05-04T06:29:24Z")

</div>

Just to elaborate on how this works, I’ve tried to make something easy enough that you can do it on your phone.

The prominent link on the README will just ask you to provide a code snippet for a package, e.g.

 ![image](https://global.discourse-cdn.com/julialang/original/3X/8/5/85070258afc99c1169b1c032d018702c09a56c68.png)

That’s it! No need to clone the repo, etc.

The minimum Julia version will be automatically determined and a PR created. If If you’re a maintainer of the package (author or member of parent org) in question, it will also be merged, e.g. [New task: HiddenMarkovModels, Estimate HMM with Baum-Welch by github-actions[bot] · Pull Request #103 · tecosaur/Julia-TTFX-Snippets · GitHub](https://github.com/tecosaur/Julia-TTFX-Snippets/pull/103)

Note that tasks are run _without_ network access and should not create any files outside of temp/cache directories.

---

<div class="post-metadata">

**Author:** ![hz-xiaxz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hz-xiaxz/32/209585_2.png) [@hz-xiaxz](https://discourse.julialang.org/u/hz-xiaxz)\
**Post date:** [October 13, 2025, 7:19am UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/52 "2025-10-13T07:19:15Z")

</div>

Any sign that this problem might be alleviate during the 1.12 release cycle?

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [October 13, 2025, 7:29am UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/53 "2025-10-13T07:29:28Z")

</div>

The point is, startup time is not well defined, and we have no good, reproducible test cases for the pre-compile time.

The package load time of 1.12 is pretty good. The pre-compile time is not. I would open an issue if I had a good test case.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [October 13, 2025, 8:49am UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/54 "2025-10-13T08:49:32Z")

</div>

> [@ufechner7](#):
>
> The point is, startup time is not well defined, and we have no good, reproducible test cases for the pre-compile time.

We’re getting them:

- Collected in [GitHub - tecosaur/Julia-TTFX-Snippets: A collection of TTFX workloads for Julia packages, for longitudinal performance testing.](https://github.com/tecosaur/Julia-TTFX-Snippets)
- Run by [GitHub - JuliaEcosystemBenchmarks/julia-ecosystem-benchmarks](https://github.com/JuliaEcosystemBenchmarks/julia-ecosystem-benchmarks)

@Krastanov has been good enough to get the runner going, and has put together some plots:

 ![BaseDirs_Project_Path](https://global.discourse-cdn.com/julialang/original/3X/e/2/e2096e580a1d9adab119230e60a674681ccf3834.png)  
Funnily enough just a few days ago Stefan and I spoke about making a more user-friendly way of tracking shift across Julia versions, and for a single package, across the different performance metrics.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [October 13, 2025, 9:02am UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/55 "2025-10-13T09:02:06Z")

</div>

Thanks for sharing these plots!

I find it a bit hard, though, to find the version I am looking for because some of the colors are very similar. And is the specification (OS, CPU, RAM, HD) of the runner known?

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [October 13, 2025, 9:03am UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/56 "2025-10-13T09:03:28Z")

</div>

> [@ufechner7](#):
>
> I find it a bit hard, though, to find the version I am looking for because some of the colors are very similar.

That’s why mentioned that a more user-friendly way of presenting the data is on the todo list (please send help, I think my todo list is collapsing under the weight of my ambition) 🙂

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [October 13, 2025, 9:04am UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/57 "2025-10-13T09:04:41Z")

</div>

Agreed - it feels like this would make more sense as a box and whiskers plot with release numbers as x axis ticklabels.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [October 13, 2025, 9:09am UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/58 "2025-10-13T09:09:04Z")

</div>

Is the data available as a CSV file or in a similar format? Then all of us could try to make improved plots.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [October 13, 2025, 9:20am UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/59 "2025-10-13T09:20:26Z")

</div>

I’m not familiar with what’s actually run: Stefan set it up not me.

However, all of the files involved in collecting data, saving it, and plotting it are in the root directory of [GitHub - JuliaEcosystemBenchmarks/julia-ecosystem-benchmarks](https://github.com/JuliaEcosystemBenchmarks/julia-ecosystem-benchmarks/tree/master).

For instance, see `ttfx_snippets_gather_data.jl`and `make_and_commit_plots.sh`. The basic understanding of the structure I have is that there’s a minimal container built (see: `Dockerfile`, basically debian 12 + juliaup), and then the `.sh` scripts are run, which in turn call the `.jl` scripts.

What I mentioned to Stefan a few days ago is the idea of committing a structured set of CSVs, making them accesssible via `GET` requests to a `raw.githubcontent.com/...` URL, and replacing the pre-generated plots with a basic interactive page where you can select what you’re comparing (and \<insert javascript framework\> will fetch and plot the data).

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [October 13, 2025, 12:31pm UTC](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/60 "2025-10-13T12:31:17Z")

</div>

The data seems to be written to a different branch

> <https://github.com/JuliaEcosystemBenchmarks/julia-ecosystem-benchmarks/blob/jeb_logs/data/Julia-TTFX-Snippets/ttfx_snippets_data.csv>

[Previous page](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343.md?page=2)

[Next page](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343.md?page=4)