Julia 1.11 beta high latency

ufechner7 · April 11, 2024, 7:30am

I had a quick test of the beta today, and found it to be 20% to 70% slower than Julia 1.10. Is this to be expected?

Datseris · April 11, 2024, 7:53am

Given how many other posts I’ve read saying how much faster things are in 1.11 due to the new Memory design, I would wager “no, it is not expected”.

Can you perhaps provide some examples, preferably as MWE?

ufechner7 · April 11, 2024, 8:33am

Installation of the example:

mkdir perf
cd perf
mkdir data
julia --project="."
using Pkg
pkg"add KiteUtils"
exit()

Julia 1.10.2
julia --project

First test:

@time using KiteUtils
  0.646830 seconds (1.46 M allocations: 90.701 MiB, 7.00% gc time, 26.15% compilation time: 78% of which was recompilation)

Second test:

using KiteUtils
KiteUtils.copy_settings()
@time KiteUtils.test(true);
  4.052462 seconds (15.21 M allocations: 1.005 GiB, 11.06% gc time, 99.87% compilation time)

Julia 1.11.0-beta1

@time using KiteUtils
  1.091960 seconds (4.58 M allocations: 242.670 MiB, 11.99% gc time, 50.97% compilation time: 79% of which was recompilation)

68% slower

using KiteUtils
KiteUtils.copy_settings()
@time KiteUtils.test(true);
  5.057151 seconds (26.70 M allocations: 1.330 GiB, 4.21% gc time, 99.89% compilation time)

24% slower

roflmaostc · April 11, 2024, 9:01am

It seems like compilation time increased?

nilshg · April 11, 2024, 9:16am

Maybe relevant:

github.com/JuliaLang/julia

v1.11 Takes longer to compile, generates 50% larger cache files and loads slower

opened 01:35AM - 03 Mar 24 UTC

joa-quim

performance regression

THe compile time is quite variable but the differences are comparable to this. …``` | | |_| | | | (_| | | Version 1.10.2 (2024-03-01) _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |__/ | julia> cd("c:/v"); @time using GMT Precompiling GMT 1 dependency successfully precompiled in 48 seconds. 87 already precompiled. 49.657821 seconds (4.51 M allocations: 328.234 MiB, 0.21% gc time, 1.84% compilation time) ``` ``` | | |_| | | | (_| | | Version 1.11.0-alpha1 (2024-03-01) _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |__/ | julia> cd("c:/v"); @time using GMT Precompiling GMT 1 dependency successfully precompiled in 60 seconds. 112 already precompiled. 61.731323 seconds (4.22 M allocations: 263.147 MiB, 0.35% gc time, 1.69% compilation time: 15% of which was recompilation) ``` v1.11 cache file -> ~89.5 MB v1.10 -> ~59.5 Mb Load times ``` | | |_| | | | (_| | | Version 1.10.2 (2024-03-01) _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |__/ | julia> @time_imports using GMT ┌ 2.7 ms SuiteSparse_jll.__init__() 25.6 ms SuiteSparse_jll 85.64% compilation time ┌ 5.0 ms SparseArrays.CHOLMOD.__init__() 98.93% compilation time 123.1 ms SparseArrays 3.99% compilation time 0.7 ms Statistics 0.2 ms DataValueInterfaces 0.6 ms DataAPI 0.2 ms IteratorInterfaceExtensions 0.2 ms TableTraits 6.3 ms Tables 0.2 ms Reexport 12.1 ms Preferences 0.3 ms PrecompileTools 5.7 ms StringManipulation 10.3 ms Crayons 0.6 ms LaTeXStrings 63.7 ms PrettyTables ┌ 17.4 ms GMT.Gdal.__init__() ├ 16.6 ms GMT.__init__() 319.5 ms GMT ``` ``` | | |_| | | | (_| | | Version 1.11.0-alpha1 (2024-03-01) _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |__/ | julia> @time_imports using GMT 0.7 ms Statistics 0.3 ms DataValueInterfaces 0.5 ms DataAPI 0.2 ms IteratorInterfaceExtensions 0.2 ms TableTraits 6.8 ms Tables 0.4 ms Reexport 8.9 ms Preferences 0.4 ms PrecompileTools 5.8 ms StringManipulation 11.7 ms Crayons 0.6 ms LaTeXStrings 69.4 ms PrettyTables ┌ 18.5 ms GMT.Gdal.__init__() ├ 108.3 ms GMT.__init__() 88.15% compilation time (100% recompilation) 814.4 ms GMT 54.63% compilation time (59% recompilation) ```

roflmaostc · April 11, 2024, 9:41am

I’m a bit surprised to not see any discussion on the issues.
Are those problems being tackled?

Palli · April 11, 2024, 10:09am

It’s on the 1.11 milestone, so it will not be ignored. It has 16 regressions (41% of the 39 open issues on the milestone, though only 8 also marked performance).

In total there are 39 regressions since not all are marked on the milestone.

I wouldn’t worry too much about regressions, this is only a beta, the first one, not even rc1.

I’m though not sure if these are many regression, or unusually many for Julia even. I’m very involved in open source now, at least follow very well what’s happening with JuliaLang, and many packages, but Julia is the first and still only language I’m involved with at that level. I barely follow other language such as Python, so I can’t say if Julia has unusually many regressions. Also Julia is like Python + NumPy at least if you want to compare, but conversely Python has way more in its standard library for non-numerical (not missing for Julia necessarily, just found in packages), so it’s not easy to compare stats of languages.

I think the regressions are a positive, in a sense, since to me it feels like much interesting work is being done, e.g. excellent work on the GC recently. Then you expect some regressions. And it’s not too hard to fix, you just bisect, and you at least have the option to revert the offending change. That’s simple, though you may not want to, and then it may be harder to fix.

Some of the intriguing issues I see looking at the list, note some only marked performance, not regression:

github.com/JuliaLang/julia

Can we speed up objectid using faster hashing

opened 01:29PM - 07 Dec 23 UTC

Zentrik

performance

I have an `IdDict` with `SimpleVector` keys, I see a lot of time is spent callin…g `jl_object_id_` to hash the keys, specifically [int64hash](https://github.com/JuliaLang/julia/blob/856e1120a8f255e4907a05f0146a2026a6665dd4/src/support/hashing.c#L28), which is called to hash any 64 bit elements in the key and it's also called in [bitmix](https://github.com/JuliaLang/julia/blob/856e1120a8f255e4907a05f0146a2026a6665dd4/src/support/hashing.h#L29). It seems `jl_object_id_` is mainly used for hashtables like `IdDict` and `IdSet` in which case would it be fine to switch to something like [FxHasher](https://docs.rs/rustc-hash/latest/rustc_hash/) (used in the rust compiler) to replace both `bitmix` and `int64hash`? Replacing `bitmix` in `hash_svec` with FxHasher gives a 10-100x improvement on `@benchmark hash(x) setup=x=Core.svec(rand(Int64, 10^n)...)` for n in 1 to 6. However, FxHasher for 64 bit ints is just a multiplication by a constant, though if objectid is only used in internal hashtables, like `IdDict`, I think this should be fine. Whilst objectid is used to hash general objects on the Julia side, its passed through a hash so I don't think it should affect `hash` in Julia too much. Alternatively, we could try using the non AES intrinsic version of ahash, this should still be much faster than what we have currently and higher quality than FxHasher. Again, ahash says it's specifically for hashtables. Alternatively, to replace only `int64hash`, [Squirrel3](https://gist.github.com/Zentrik/b5053668935102719ebf619944ce3e6a) is quite simple and seems to be reasonably fast (about 1.5-2x faster than int64hash) and should have higher quality hashes than FxHasher. Does it seem reasonable to make a change along these lines? I see that currently `int64hash` is [invertible](https://gist.github.com/lh3/974ced188be2f90422cc), do we care about this? I don't see it being used anywhere. This would also speed up my robinhood implementation of `IdDict`.

github.com/JuliaLang/julia

3.5x regression on bounds checking in BaseBenchmarks due to LLVM bump to 15.0.7+10 in #52405

opened 10:42PM - 11 Mar 24 UTC

Zentrik

performance regression

#52405 bumps LLVM but it seems to have only been a small patch https://github.co…m/JuliaLang/llvm-project/compare/julia-15.0.7-9...julia-15.0.7-10. It caused a 3.5x regression on `BaseBenchmarks.SUITE[["array", "index", ("sumelt_boundscheck", "BaseBenchmarks.ArrayBenchmarks.ArrayLF{Int32, 2}")]]` which is still present (slightly worse) on master. [Long run graph](http://tealquaternion.camdvr.org/graphs.html?start=2023-11-07&benchmark=array.index.%28sumelt_boundscheck%2C+BaseBenchmarks.ArrayBenchmarks.ArrayLF%7BInt32%2C+2%7D%29&profile=opt&scenario=full&stat=min-wall-time&kind=raw) and the [Nanosoldier Report](https://github.com/JuliaCI/NanosoldierReports/blob/master/benchmark/by_date/2023-12/13/report.md). Feel free to close if not an issue.

github.com/JuliaLang/julia

GC tuning_factor might need more tuning

opened 10:12PM - 26 Feb 24 UTC

d-netto

performance GC

I think there might be some work left on getting a suitable value for the tuning…_factor in src/gc.c . I bumped it from 2e4 to 2e5 and got more than a 2x reduction in GC times with maximum heap size decreasing as well. This is for append.jl FWIW. - 2e4: ``` ┌─────────┬────────────┬─────────┬───────────┬────────────┬──────────────┬───────────────────┬──────────┬────────────┐ │ │ total time │ gc time │ mark time │ sweep time │ max GC pause │ time to safepoint │ max heap │ percent gc │ │ │ ms │ ms │ ms │ ms │ ms │ us │ MB │ % │ ├─────────┼────────────┼─────────┼───────────┼────────────┼──────────────┼───────────────────┼──────────┼────────────┤ │ minimum │ 952 │ 280 │ 177 │ 103 │ 50 │ 95 │ 3698 │ 29 │ │ median │ 959 │ 286 │ 177 │ 107 │ 50 │ 101 │ 3700 │ 30 │ │ maximum │ 964 │ 286 │ 179 │ 108 │ 56 │ 103 │ 4032 │ 30 │ │ stdev │ 6 │ 3 │ 1 │ 3 │ 4 │ 4 │ 192 │ 0 │ └─────────┴────────────┴─────────┴───────────┴────────────┴──────────────┴───────────────────┴──────────┴────────────┘ ``` - 2e5: ``` ┌─────────┬────────────┬─────────┬───────────┬────────────┬──────────────┬───────────────────┬──────────┬────────────┐ │ │ total time │ gc time │ mark time │ sweep time │ max GC pause │ time to safepoint │ max heap │ percent gc │ │ │ ms │ ms │ ms │ ms │ ms │ us │ MB │ % │ ├─────────┼────────────┼─────────┼───────────┼────────────┼──────────────┼───────────────────┼──────────┼────────────┤ │ minimum │ 790 │ 116 │ 39 │ 77 │ 40 │ 54 │ 3246 │ 15 │ │ median │ 799 │ 120 │ 39 │ 81 │ 41 │ 63 │ 3249 │ 15 │ │ maximum │ 802 │ 126 │ 40 │ 86 │ 50 │ 67 │ 3249 │ 16 │ │ stdev │ 6 │ 5 │ 1 │ 4 │ 5 │ 7 │ 2 │ 1 │ └─────────┴────────────┴─────────┴───────────┴────────────┴──────────────┴───────────────────┴──────────┴────────────┘ ```

github.com/JuliaLang/julia

simd performance regression in 1.10+rc1 relative to 1.9.4

opened 12:43AM - 26 Nov 23 UTC

lmiq

performance regression simd

Related to this discourse thread: https://discourse.julialang.org/t/trivial-p…ort-from-c-rust-to-julia/105787/46?u=lmiq The code posted here runs much faster in 1.9.4 than in 1.10+rc1: ``` % time julia +1.9 --startup-file=no lv2_c.jl 0 1000 0.01 200 typeof(rng) = MarsagliaRng Mean IS=0.04462564329047784 OOS=-0.0031496723033457423 Bias=0.047775315593823586 11.083806 seconds (169.93 k allocations: 16.234 MiB, 1.22% compilation time) real 0m11,553s user 0m11,605s sys 0m0,318s % time julia +1.10 --startup-file=no lv2_c.jl 0 1000 0.01 200 typeof(rng) = MarsagliaRng Mean IS=0.04462564329047784 OOS=-0.0031496723033457423 Bias=0.047775315593823586 42.888539 seconds (168.42 k allocations: 16.271 MiB, 0.38% compilation time) real 0m43,412s user 0m43,523s sys 0m0,291s ``` The issue is related in how the loop that starts with `for i in (ilong:ncases-2)` is lowered. If one adds `@simd` to that loop, performance become: ``` % time julia +1.9 --startup-file=no lv2_d.jl 0 1000 0.01 200 typeof(rng) = MarsagliaRng Mean IS=0.04462564329047785 OOS=-0.0031496723033457423 Bias=0.04777531559382359 4.091737 seconds (179.15 k allocations: 16.861 MiB, 3.62% compilation time) real 0m4,544s user 0m4,618s sys 0m0,314s % time julia +1.10 --startup-file=no lv2_d.jl 0 1000 0.01 200 typeof(rng) = MarsagliaRng Mean IS=0.04462564329047785 OOS=-0.0031496723033457423 Bias=0.04777531559382359 3.953665 seconds (177.21 k allocations: 16.855 MiB, 4.40% compilation time) real 0m4,380s user 0m4,486s sys 0m0,297s ``` # The code: ```julia using Random using Printf struct ParamsResult short_term::Int long_term::Int performance::Float64 end mutable struct MarsagliaRng q::Vector{UInt32} carry::UInt32 mwc256_initialized::Bool mwc256_seed::Int32 i::UInt8 function MarsagliaRng(seed::Vector{UInt8}) q = zeros(UInt32, 256) carry = 362436 mwc256_initialized = false mwc256_seed = reinterpret(Int32, seed)[1] i = 255 new(q, carry, mwc256_initialized, mwc256_seed, i) end end # Default constructor using system time for seeding function MarsagliaRng() ts_nano = UInt8(nanosecond(now())) seed = [ts_nano, ts_nano >>> 8, ts_nano >>> 16, ts_nano >>> 24] MarsagliaRng(seed) end # Random number generator functions function next_u32(rng::MarsagliaRng)::UInt32 a = UInt64(809430660) if !rng.mwc256_initialized j = UInt32(rng.mwc256_seed) c1 = UInt32(69069) c2 = UInt32(12345) rng.mwc256_initialized = true for k in 1:256 j = (c1 * j + c2) % UInt32 rng.q[k] = j end end rng.i = (rng.i + 1) % 256 t = a * UInt64(rng.q[rng.i+1]) + UInt64(rng.carry) rng.carry = (t >> 32) rng.q[rng.i+1] = t % UInt32 return rng.q[rng.i+1] end function next_u64(rng::MarsagliaRng)::UInt64 UInt64(next_u32(rng)) end # EDIT: extend Base.rand rather than shadowing it. # Sample function for generating a Float64 function Base.rand(rng::MarsagliaRng)::Float64 mult = 1.0 / typemax(UInt32) mult * next_u32(rng) end function opt_params(which, ncases::Integer, x::Vector{Float64}) FloatT = eltype(x) xs = cumsum(x) best_perf = typemin(FloatT) ibestshort = 0 ibestlong = 0 small_float = 1e-60 for ilong in 2:199 for ishort in 1:(ilong-1) total_return = zero(FloatT) win_sum = small_float lose_sum = small_float sum_squares = small_float short_sum = zero(FloatT) long_sum = zero(FloatT) let i = ilong - 1 short_sum = xs[i+1] - xs[i-ishort + 2 - 1] long_sum = xs[i+1] short_mean = short_sum / ishort long_mean = long_sum / ilong ret = (short_mean > long_mean) ? x[i+2] - x[i+1] : (short_mean < long_mean) ? x[i+1] - x[i+2] : zero(FloatT) total_return += ret sum_squares += ret^2 if ret > zero(FloatT) win_sum += ret else lose_sum -= ret end end # General case; i != ilong - 1 # @simd for i in (ilong:ncases-2) # for fast code for i in (ilong:ncases-2) x1 = x[i+1] #short_sum += x1 - x[i-ishort+1] #long_sum += x1 - x[i-ilong+1] short_sum = xs[i+1] - xs[i-ishort + 1] long_sum = xs[i+1] - xs[i-ilong + 1] short_mean = short_sum / ishort long_mean = long_sum / ilong ret = (short_mean > long_mean) ? x[i+2] - x1 : (short_mean < long_mean) ? x1 - x[i+2] : zero(FloatT) total_return += ret sum_squares += ret^2 if ret > zero(FloatT) win_sum += ret else lose_sum -= ret end end if which == 0 total_return /= (ncases - ilong) if total_return > best_perf best_perf = total_return ibestshort = ishort ibestlong = ilong end elseif which == 1 pf = win_sum / lose_sum if pf > best_perf best_perf = pf ibestshort = ishort ibestlong = ilong end elseif which == 2 total_return /= (ncases - ilong) sum_squares /= (ncases - ilong) sum_squares -= total_return^2 sr = total_return / (sqrt(sum_squares) + 1e-8) if sr > best_perf best_perf = sr ibestshort = ishort ibestlong = ilong end end end end ParamsResult(ibestshort, ibestlong, best_perf) end function test_system(ncases::Integer, x::Vector{Float64}, short_term::Integer, long_term::Integer) FloatT = eltype(x) sum1 = zero(FloatT) for i in (long_term-1:ncases-2) short_mean = sum(@view x[i-short_term+2:i+1]) / short_term long_mean = sum(@view x[i-long_term+2:i+1]) / long_term if short_mean > long_mean sum1 += x[i+2] - x[i+1] elseif short_mean < long_mean sum1 -= x[i+2] - x[i+1] end end sum1 / (ncases - long_term) end function main() if length(ARGS) < 4 println("Usage: julia script.jl <which> <ncases> <trend> <nreps>") return end which = parse(Int, ARGS[1]) ncases = parse(Int, ARGS[2]) save_trend = parse(Float64, ARGS[3]) nreps = parse(Int, ARGS[4]) optimize(which, ncases, save_trend, nreps) end function optimize(which::Integer, ncases::Integer, save_trend::Float64, nreps::Integer, rng=MarsagliaRng(UInt8.([33, 0, 0, 0]))) @show typeof(rng) _optimize(rng, which, ncases, save_trend, nreps) end function _optimize(rng, which::Integer, ncases::Integer, save_trend::Float64, nreps::Integer) print_results = false FloatT = typeof(save_trend) x = zeros(FloatT, ncases) is_mean = zero(FloatT) oos_mean = zero(FloatT) futures = Task[] for irep in 1:nreps # Generate in-sample trend = save_trend x[1] = zero(FloatT) for i in 2:ncases if (i - 1) % 50 == 0 trend = -trend end x[i] = x[i-1] + trend + rand(rng) + rand(rng) - rand(rng) - rand(rng) end x_optim = copy(x) # Generate out-of-sample trend = save_trend x[1] = zero(FloatT) for i in 2:ncases if (i - 1) % 50 == 0 trend = -trend end x[i] = x[i-1] + trend + rand(rng) + rand(rng) - rand(rng) - rand(rng) end x_oos = copy(x) future = Threads.@spawn begin params_result = opt_params(which, ncases, x_optim) oos_perf = test_system(ncases, x_oos, params_result.short_term, params_result.long_term) (params_result, oos_perf) end push!(futures, future) end for (irep, future) in enumerate(futures) params_result::ParamsResult, oos_perf::Float64 = fetch(future) is_mean += params_result.performance oos_mean += oos_perf if print_results @printf("%3d: %3d %3d %8.4f %8.4f (%8.4f)\n", irep, params_result.short_term, params_result.long_term, params_result.performance, oos_perf, params_result.performance - oos_perf) end end is_mean /= nreps oos_mean /= nreps println("Mean IS=$is_mean OOS=$oos_mean Bias=$(is_mean - oos_mean)") end @time main() ```

github.com/JuliaLang/julia

Regression in sparse-dense multiplication

opened 07:59PM - 12 Nov 23 UTC

dkarrasch

performance regression linear algebra

Running this piece of code, I get ```julia using LinearAlgebra, SparseArrays…, BenchmarkTools function _spmatmul!(C, A, B, α, β) size(A, 2) == size(B, 1) || throw(DimensionMismatch()) size(A, 1) == size(C, 1) || throw(DimensionMismatch()) size(B, 2) == size(C, 2) || throw(DimensionMismatch()) nzv = nonzeros(A) rv = rowvals(A) β != one(β) && LinearAlgebra._rmul_or_fill!(C, β) for k in 1:size(C, 2) @inbounds for col in 1:size(A, 2) αxj = B[col,k] * α for j in nzrange(A, col) C[rv[j], k] += nzv[j]*αxj end end end C end n = 1000; A = sparse(Float64, I, n, n); b = Vector{Float64}(undef, n); x = ones(n); @btime _spmatmul!($b, $A, $x, true, false); # 1.703 μs (0 allocations: 0 bytes) on v1.9.3 # 2.161 μs (0 allocations: 0 bytes) on v1.10.0-rc1 # 2.945 μs (0 allocations: 0 bytes) on an 11 days old master ``` Multiplication dispatch has changed over the 1.9 to 1.10 transition, but this is nothing but the barebone mutliplication code that we used to have "ever since", so without character processing and all that. And since this is not calling high-level functions, the issue must be outside of SparseArrays.jl, AFAIU. x-ref https://github.com/JuliaSparse/SparseArrays.jl/issues/469

While I’m here, I’ve been thinking do languages, at least Julia return memory to the OS, and I found my answer, but only for Java:

Some GC implementations actively support heap shrinking. Heap shrinking is the process of releasing back the excess memory from heap to OS for optimal resource usage.

For example, Parallel GC doesn’t release unused memory back to the OS readily. On the other hand, some GCs analyze the memory consumption and determine accordingly to release some free memory from the heap. G1, Serial, Shenandoah, and Z GCs support heap shrinking.

For those who’ve not seem this/been following AI music generation, or miss out, I felt I had to post this here, since hilarious, and on MIT license, also good vocals. Maybe open an issue, Julia should link this not the license itself? Or make a happy version since we like the license. https://twitter.com/goodside/status/1775713487529922702

mkitti · April 11, 2024, 10:55am

I have been trying to tell people about this for a while. Julia 1.11 is going to be slower than Julia 1.10 for a number of reasons. There are a lot of big changes landing in this release. Besides the Memory change, more packages have been kicked out of the system image.

If you a performance sensitive user, Julia 1.10 might be your version for a while as the ecosystem adapts to these changes.

joa-quim · April 11, 2024, 10:59am

OK, comprehensible, but than, sorry, why releasing 1.11 in that state?

lassepe · April 11, 2024, 11:36am

Training of some of my Flux models is also about 20% slower (irrespective of load times or compilation times). I guess this would also be related to the Memory change (?)

@mkitti can you share some info on which kind of changes package developers should consider to get back to 1.10 speed?

jishnub · April 11, 2024, 11:43am

It’s not released yet, and work is underway to address regressions before the actual release. The beta versions are meant for users and package developers to point out issues with the upcoming release. It would actually be good if new regressions or breakages are reported to the issue tracker.

joa-quim · April 11, 2024, 12:30pm

You didn’t quote the full story.

EDIT: Sorry for misunderstanding, this was a reply to the jishnub’s comment above.

mkitti · April 11, 2024, 12:59pm

Here’s a small part of it:

kristoffer.carlsson · April 11, 2024, 1:12pm

Most of the load time regression seems to come from TimeZones:

julia> @time using TimeZones
  0.096659 seconds (402.11 k allocations: 18.721 MiB, 20.38% compilation time)

julia> VERSION
v"1.10.2"

vs

julia> @time using TimeZones
  0.456108 seconds (3.32 M allocations: 165.423 MiB, 13.80% gc time, 68.63% compilation time: 90% of which was recompilation)

julia> VERSION
v"1.11.0-beta1"

Particularly, the compilation of the __init__ method in the package.

mrufsvold · April 11, 2024, 1:50pm

Maybe 1.11 is in a time zone that’s 0.4 seconds ahead?

mbauman · April 11, 2024, 2:10pm

Yes, this is precisely why betas are released and is the kind of feedback that’s needed to get to make releases better. If we knew that there weren’t problems/regressions, this would’ve been tagged as a release candidate. The beta simply means that there won’t be net-new features added anymore.

Please don’t feel sheepish about posting regressions in the beta — or, really, ever! Sure, some might already be known, but the more minimal you can make the example the more likely someone might be able to identify that.

Julia v1.11 does incorporate some pretty big changes, so all the more reason to check out the betas! The more folks that do, the better the release will be.

nilshg · April 11, 2024, 2:16pm

Is @mkitti’s position above the official position of the core devs? I.e. Julia 1.11 would be released despite material performance regression, because the changes introduced warrant this and performance sensitive users shouldn’t upgrade?

mbauman · April 11, 2024, 4:36pm

9 posts were split to a new topic: Increase in allocations with Julia v1.11-beta

mkitti · April 11, 2024, 2:20pm

Part of it is that a fair amount of work needs to be done on the package side to take advantage of Julia 1.11’s features.

For what I cited above, Pkg.jl needs to be adapted to work well as a regular package outside the system image. One concern that I hope is addressed before release is invalidations of Base.BinaryPlatforms method by Pkg.BinaryPlatforms that can lead to a fair amount of recompilation when using certain JLLs or artifacts.

For other packages, loading Pkg is no longer relatively free, so some packages that do this may want to consider other routes. Base has some rudimentary functionality that could be used instead of Pkg. With Julia 1.11, there is a clearer incentive to use those over Pkg APIs for lightweight work. Similar advice goes for other former standard library packages which were in the system image but are no longer there in Julia 1.11.

While some of the changes need to be adapted to, ultimately they should eventually enable everything else to be faster due to the system image being smaller.

mkitti · April 11, 2024, 2:28pm

I do not speak for the “core devs”, which is a nebulous concept. However, history shows us that performance is not guaranteed to monotically increase over time with each release. We saw some performance regressions in Julia 1.7 and even Julia 1.8 before performance gains were consolidated in Julia 1.9 and Julia 1.10. Sometimes performance needs to get worse before it gets better.

I think Julia 1.10 is a strong candidate for a Long Term Support release. That seems to be the general sentiment as well. This also partly is why a lot of more drastic changes are landing in Julia 1.11 since I think they were deferred until after the LTS candidate.

As Matt says, we should really take advantage of the beta to try to address some performance regresssions. However, I also think some needed improvements might not come until Julia 1.12 or later.