Julia 1.11 beta high latency

I had a quick test of the beta today, and found it to be 20% to 70% slower than Julia 1.10. Is this to be expected?

Given how many other posts I’ve read saying how much faster things are in 1.11 due to the new Memory design, I would wager β€œno, it is not expected”.

Can you perhaps provide some examples, preferably as MWE?

Installation of the example:

mkdir perf
cd perf
mkdir data
julia --project="."
using Pkg
pkg"add KiteUtils"

Julia 1.10.2
julia --project

First test:

@time using KiteUtils
  0.646830 seconds (1.46 M allocations: 90.701 MiB, 7.00% gc time, 26.15% compilation time: 78% of which was recompilation)

Second test:

using KiteUtils
@time KiteUtils.test(true);
  4.052462 seconds (15.21 M allocations: 1.005 GiB, 11.06% gc time, 99.87% compilation time)

Julia 1.11.0-beta1

@time using KiteUtils
  1.091960 seconds (4.58 M allocations: 242.670 MiB, 11.99% gc time, 50.97% compilation time: 79% of which was recompilation)

68% slower

using KiteUtils
@time KiteUtils.test(true);
  5.057151 seconds (26.70 M allocations: 1.330 GiB, 4.21% gc time, 99.89% compilation time)

24% slower

It seems like compilation time increased?

1 Like

Maybe relevant:


I’m a bit surprised to not see any discussion on the issues.
Are those problems being tackled?

It’s on the 1.11 milestone, so it will not be ignored. It has 16 regressions (41% of the 39 open issues on the milestone, though only 8 also marked performance).

In total there are 39 regressions since not all are marked on the milestone.

I wouldn’t worry too much about regressions, this is only a beta, the first one, not even rc1.

I’m though not sure if these are many regression, or unusually many for Julia even. I’m very involved in open source now, at least follow very well what’s happening with JuliaLang, and many packages, but Julia is the first and still only language I’m involved with at that level. I barely follow other language such as Python, so I can’t say if Julia has unusually many regressions. Also Julia is like Python + NumPy at least if you want to compare, but conversely Python has way more in its standard library for non-numerical (not missing for Julia necessarily, just found in packages), so it’s not easy to compare stats of languages.

I think the regressions are a positive, in a sense, since to me it feels like much interesting work is being done, e.g. excellent work on the GC recently. Then you expect some regressions. And it’s not too hard to fix, you just bisect, and you at least have the option to revert the offending change. That’s simple, though you may not want to, and then it may be harder to fix.

Some of the intriguing issues I see looking at the list, note some only marked performance, not regression:

While I’m here, I’ve been thinking do languages, at least Julia return memory to the OS, and I found my answer, but only for Java:

Some GC implementations actively support heap shrinking. Heap shrinking is the process of releasing back the excess memory from heap to OS for optimal resource usage.

For example, Parallel GC doesn’t release unused memory back to the OS readily. On the other hand, some GCs analyze the memory consumption and determine accordingly to release some free memory from the heap. G1, Serial, Shenandoah, and Z GCs support heap shrinking.

For those who’ve not seem this/been following AI music generation, or miss out, I felt I had to post this here, since hilarious, and on MIT license, also good vocals. Maybe open an issue, Julia should link this not the license itself? :slight_smile: Or make a happy version since we like the license. https://twitter.com/goodside/status/1775713487529922702


I have been trying to tell people about this for a while. Julia 1.11 is going to be slower than Julia 1.10 for a number of reasons. There are a lot of big changes landing in this release. Besides the Memory change, more packages have been kicked out of the system image.

If you a performance sensitive user, Julia 1.10 might be your version for a while as the ecosystem adapts to these changes.


OK, comprehensible, but than, sorry, why releasing 1.11 in that state?


Training of some of my Flux models is also about 20% slower (irrespective of load times or compilation times). I guess this would also be related to the Memory change (?)

@mkitti can you share some info on which kind of changes package developers should consider to get back to 1.10 speed?


It’s not released yet, and work is underway to address regressions before the actual release. The beta versions are meant for users and package developers to point out issues with the upcoming release. It would actually be good if new regressions or breakages are reported to the issue tracker.


You didn’t quote the full story.

EDIT: Sorry for misunderstanding, this was a reply to the jishnub’s comment above.

Here’s a small part of it:

1 Like

Most of the load time regression seems to come from TimeZones:

julia> @time using TimeZones
  0.096659 seconds (402.11 k allocations: 18.721 MiB, 20.38% compilation time)

julia> VERSION


julia> @time using TimeZones
  0.456108 seconds (3.32 M allocations: 165.423 MiB, 13.80% gc time, 68.63% compilation time: 90% of which was recompilation)

julia> VERSION

Particularly, the compilation of the __init__ method in the package.


Maybe 1.11 is in a time zone that’s 0.4 seconds ahead?


Yes, this is precisely why betas are released and is the kind of feedback that’s needed to get to make releases better. If we knew that there weren’t problems/regressions, this would’ve been tagged as a release candidate. The beta simply means that there won’t be net-new features added anymore.

Please don’t feel sheepish about posting regressions in the beta β€” or, really, ever! Sure, some might already be known, but the more minimal you can make the example the more likely someone might be able to identify that.

Julia v1.11 does incorporate some pretty big changes, so all the more reason to check out the betas! The more folks that do, the better the release will be.


Is @mkitti’s position above the official position of the core devs? I.e. Julia 1.11 would be released despite material performance regression, because the changes introduced warrant this and performance sensitive users shouldn’t upgrade?


9 posts were split to a new topic: Increase in allocations with Julia v1.11-beta

Part of it is that a fair amount of work needs to be done on the package side to take advantage of Julia 1.11’s features.

For what I cited above, Pkg.jl needs to be adapted to work well as a regular package outside the system image. One concern that I hope is addressed before release is invalidations of Base.BinaryPlatforms method by Pkg.BinaryPlatforms that can lead to a fair amount of recompilation when using certain JLLs or artifacts.

For other packages, loading Pkg is no longer relatively free, so some packages that do this may want to consider other routes. Base has some rudimentary functionality that could be used instead of Pkg. With Julia 1.11, there is a clearer incentive to use those over Pkg APIs for lightweight work. Similar advice goes for other former standard library packages which were in the system image but are no longer there in Julia 1.11.

While some of the changes need to be adapted to, ultimately they should eventually enable everything else to be faster due to the system image being smaller.

1 Like

I do not speak for the β€œcore devs”, which is a nebulous concept. However, history shows us that performance is not guaranteed to monotically increase over time with each release. We saw some performance regressions in Julia 1.7 and even Julia 1.8 before performance gains were consolidated in Julia 1.9 and Julia 1.10. Sometimes performance needs to get worse before it gets better.

I think Julia 1.10 is a strong candidate for a Long Term Support release. That seems to be the general sentiment as well. This also partly is why a lot of more drastic changes are landing in Julia 1.11 since I think they were deferred until after the LTS candidate.

As Matt says, we should really take advantage of the beta to try to address some performance regresssions. However, I also think some needed improvements might not come until Julia 1.12 or later.