### Author:  [@jakobnissen](https://discourse.julialang.org/u/jakobnissen)
#### Post date: [October 17, 2025, 9:11am UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/2 "2025-10-17T09:11:43Z")
I don’t think it’s true that the runtimes are getting worse in 1.12. The compiler is generally getting better over time, with better inference and more optimisations.
Performance regressions hit every language, although Julia **is** uniquely vulnerable to performance regressions among high-performance languages due to how Julia, by design, seamlessly mixes slow, dynamic code with fast statically inferred code. That makes it easy for the latter to slip into the former category undetected.
However, every release has some performance regressions which may be due to either small compiler details, or some changes in Base. It’s almost certainly one of these.
In this case, it would be helpful if you could drill down to what, precisely, is slowing down. Create a smaller, copy-pasteable example. Is something suddenly allocating? No longer SIMD’ing? These things are usually solvable.
If your question is broader: “How can I prevent performance regressions?”, then the answer is that there is no way - in any language - you can prevent them. But also, that Julia is expected to have a higher baseline of performance regressions.
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [October 17, 2025, 9:14am UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/3 "2025-10-17T09:14:36Z")
Isn’t the MWE linked above enough to reproduce?
The GitHub Actions also show this consistent slow-down across all platforms:
>
---
### Author:  [@jakobnissen](https://discourse.julialang.org/u/jakobnissen)
#### Post date: [October 17, 2025, 9:15am UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/4 "2025-10-17T09:15:47Z")
It’s enough to reproduce, but there is some work needed to drill down to what is causing the regression among all the code in your package and all its dependencies.
What would be needed in order to pin down _why_ the regression happened is some small example, ideally dependency-free, that can be copy-pasted directly into a REPL.
---
### Author:  [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [October 17, 2025, 9:16am UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/5 "2025-10-17T09:16:37Z")
Is there a real regression? I see this:
```julia-auto
julia> [@elapsed sill(γ) for _ ∈ 1:10]
10-element Vector{Float64}:
5.2e-6
2.0e-7
1.0e-7
0.0
0.0
0.0
0.0
1.0e-7
0.0
1.0e-7
julia> versioninfo()
Julia Version 1.12.0
```
and
```julia-auto
julia> [@elapsed sill(γ) for _ ∈ 1:10]
10-element Vector{Float64}:
2.0e-6
1.0e-7
0.0
0.0
1.0e-7
0.0
1.0e-7
0.0
1.0e-7
0.0
julia> versioninfo()
Julia Version 1.11.7
```
so maybe the test is just a bit flaky and too dependent on what runner it’s on?
`@btime` gives an allocation-free sub 2ns result on both 1.11 and 1.12, so looks like the function which if I see correctly is just
```julia-auto
sill(f::CompositeFunction) = sum(f.cs .* map(sill, f.fs))
```
gets constant folded.
---
### Author:  [@Benny](https://discourse.julialang.org/u/Benny)
#### Post date: [October 17, 2025, 9:24am UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/6 "2025-10-17T09:24:23Z")
> [@juliohm](#):
>
> I don’t know what else I can do to preserve the performance of packages I’ve written a long time ago. I believe all of them contain idiomatic Julia code, follow best practices, etc.
Profile on available platforms and identify code or dependencies to patch, I suppose. There are no performance guarantees across versions of source code, I don’t think that’s even feasible until you’re talking about a particular environment’s binaries on tested platforms. At the language level, you’re not only vulnerable to changes in the language implementation as this implies, you’re at the mercy of all of your dependencies. `LoopVectorization` eventually got some funding to be maintained for at least a while longer, but it was nearly deprecated for Julia v1.11 to the detriment of all the high-performance packages that relied on its loop auto-vectorization. Nobody sets out to cause performance regressions, but we do need people and resources to spot and fix anything.
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [October 17, 2025, 10:04am UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/7 "2025-10-17T10:04:45Z")
We had some conversations on Zulip where some people started to suspect `@elapsed` is buggy, and could be leaking some extra time measurements that are not captured in BenchmarkTools.jl
Given that this regression is not reproduced locally in @nilshg and others’s machines, I wonder if it has to do with GitHub setup or `@elapsed` itself.
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [October 17, 2025, 4:46pm UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/8 "2025-10-17T16:46:45Z")
More evidence in another package (TableTransforms.jl):
Actions are taking 2-3x more time to finish in Julia v1.12.
---
### Author:  [@nsajko](https://discourse.julialang.org/u/nsajko)
#### Post date: [October 17, 2025, 4:56pm UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/9 "2025-10-17T16:56:50Z")
In my experience v1.12 CI is faster than v1.11 CI, and almost as fast as v1.10 CI. Obviously performance will depend on the workload. Not sure if it’s useful to bring up such “evidence” if you’re not going to dig into what is causing it.
---
### Author:  [@mbauman](https://discourse.julialang.org/u/mbauman)
#### Post date: [October 17, 2025, 4:59pm UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/10 "2025-10-17T16:59:04Z")
The CI wall-time is surely a mix of [Startup time of 1000 packages – 53% slower in Julia 1.12 vs 1.10 - #81 by ufechner7](https://discourse.julialang.org/t/startup-time-of-1000-packages-53-slower-in-julia-1-12-vs-1-10/128343/81) (where you’re already participating) and runtime.
The most valuable thing here are concrete and isolated cases. Let’s focus this on those concrete examples where you can isolate a _runtime_ regression.
---
### Author:  [@ianshmean](https://discourse.julialang.org/u/ianshmean)
#### Post date: [October 17, 2025, 5:11pm UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/11 "2025-10-17T17:11:27Z")
@juliohm TableTransforms.jl isn’t using [GitHub - julia-actions/cache: A shortcut action to cache Julia artifacts, packages, and registries.](https://github.com/julia-actions/cache)
I recommend trying that out.
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [October 17, 2025, 5:36pm UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/12 "2025-10-17T17:36:11Z")
We missed the cache on this repo. Thanks for catching @ianshmean 👍
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [October 17, 2025, 7:44pm UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/13 "2025-10-17T19:44:42Z")
Numbers don’t improve much with the cache action:
---
### Author:  [@simsurace](https://discourse.julialang.org/u/simsurace)
#### Post date: [October 18, 2025, 11:50am UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/14 "2025-10-18T11:50:03Z")
If you just look at the list of precompile timings, you can explain most of the difference. The test run time is just 40s longer on 1.12 ubuntu vs. lts ubuntu. Sure that‘s slower and worth looking into, but not 2x.
---
### Author:  [@joa-quim](https://discourse.julialang.org/u/joa-quim)
#### Post date: [October 18, 2025, 1:21pm UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/15 "2025-10-18T13:21:19Z")
I have mentioned that before (though did not show a screenshot with the times). GMT CI times are the **double** for Julia 1.12 compared to 1.10 (12 min vs 24 min), but the big jump actually ocuured at Julia 1.11.
---
### Author:  [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)
#### Post date: [October 18, 2025, 6:46pm UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/16 "2025-10-18T18:46:50Z")
> [@juliohm](#):
>
> Numbers don’t improve much with the cache action:
We are now talking about a completely different thing from what this thread started with. Is the original note about the runtime of 1.12 resolved and this thread has pivoted to end to end CI time?
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [October 19, 2025, 7:14am UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/17 "2025-10-19T07:14:37Z")
The elapsed tests posted in the beginning of this thread are still failing in Julia v1.12 after warmup. So I don’t think the thread is solved.
Other issues related to CI are also relevant.
---
### Author:  [@ufechner7](https://discourse.julialang.org/u/ufechner7)
#### Post date: [October 19, 2025, 10:37am UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/18 "2025-10-19T10:37:56Z")
> [@juliohm](#):
>
> The elapsed tests posted in the beginning of this thread are still failing in Julia v1.12 after warmup.
Your assumption that `@elapsed` is an accurate way to measure the runtime of a function is wrong.
`@btime` is already better. Nevertheless, the performance of GitHub runners can vary widely.
Other question: Are you sure that code coverage is disabled when you run your performance tests? Code coverage testing can increase code execution time a lot in an unpredictable way.
---
### Author:  [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)
#### Post date: [October 21, 2025, 3:53pm UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/19 "2025-10-21T15:53:12Z")
> [@juliohm](#):
>
> The elapsed tests posted in the beginning of this thread are still failing in Julia v1.12 after warmup.
Okay, but you are running `@elapsed` on a shared compute resource (CI). The way to report a performance regression is to basically do:
- post the code (ideally can be copy pasted)
- run it with BenchmarkTools
- show the results on one julia version compared to another
- show that the newer julia version is worse
An `@elapsed` call in some CI log will not gather any interest.
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [October 22, 2025, 12:45am UTC](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231/20 "2025-10-22T00:45:48Z")
I could certainly replace `@elapsed` by `@btime` and will probably do that when I find some time, however, it will just hide a performance regression, which can be the result of various things as you said (CI, codecov, etc.).
Suppose Julia v1.12 runtimes have improved compared to previous versions of the language, which is a reasonable hypothesis. Still, the net effect we are seeing across various packages is undeniable: tests take twice the time to finish with Julia v1.12.
Anyone can copy/paste the MWE from the test suite and run in their own machines with Julia v1.10, v1.11 and v1.12. If the `@btime` results match, then we can attribute the issue to the `@elapsed` implementation across Julia versions or to the interaction of Julia with GitHub CI and codecov tools.
[Next page](https://discourse.julialang.org/t/worse-runtimes-in-julia-v1-12/133231.md?page=2)