What are the current plans regarding a new LTS Julia version?

The plan is to backport support for that to 1.6.

There has certainly been a lot of discussion about whether we really want 1.6 to be the LTS and note that a final decision on this has not yet been made (Although it seems fairly likely at this point). The problem is that we will (hopefully) never stop putting cool new features into releases, but at some point you just have to bite the bullet and declare an LTS, because we would really like to stop having to support 1.0 for much longer. If we waited until 1.7, there would likely just be similar calls for 1.8 to be the LTS instead.

9 Likes

Will the ability to do x.b .= y when :b is not a property of x be backported to 1.6?

No, new features are generally not backported.

4 Likes

Sounds good. DataFrames users might use this feature pretty frequently in the future (or they might not). I don’t know how much consideration should be given to that when considering the choice of LTS.

1 Like

Thanks, @simeonschaub !

It seems obvious 1.7 to me should be the next LTS, as I thought it would. We’ve waited this long, and I believe there’s consensus by now that 1.6 is a good release, so why not go with that?

So what are the pros and cons for 1.6 vs 1.7 (or later) for LTS?

It seems you could backport all of 1.7+ good features to 1.6 (you technically can for each individual one, but there are reasons you don’t want to).

I’ll first highlight some pros of Julia 1.7, from NEWS (now in HISTORY on master):

  • “The default random number generator has changed, so all random numbers will be different (even with the same seed) unless an explicit RNG object is used.” While people shouldn’t be relying on the exact numbers, it seems if might be confusing for some users to get a different stream for current and LTS. I’m not sure this affects package authors much, more what people read into the LTS term.
  • Additional functions can be handled with Compat.jl, but I’m not sure it can handle new syntax, and 1.7 has an interesting “New language features” section, e.g. “Repeated semicolons” feature, and “backslash ( \ ) before a newline”. It seems annoying to be restricted to older syntax.
  • macroexpand , @macroexpand , and @macroexpand1 no longer wrap errors in a LoadError . To reduce breakage, @test_throws has been modified so that many affected tests will still pass (#38379).” That’s the first time I see “break[age]” ever in NEWS. I don’t make my own macros, and not up-to-speed on that feature, but I see it’s implemented with e.g. C code, so again I’m not sure Compat.jl can support it.
  • “Use Libblastrampoline to pick a BLAS and LAPACK at runtime.” A very important feature going forward, would it be bad to not have in LTS? Or was this it the other way?

I could go through all of NEWS, and I guess everything counts as a pro, Julia is continually improving. So what are the potential cons for 1.7? Anything obvious to see (there)?

  • “The RepString type has been moved to the LegacyStrings.jl package.” This isn’t a big deal as few using, but it seems strange to support it in LTS, if 1.6 chosen, but not in 1.7+.
  • The middle dot · ( \cdotp U+00b7) and the Greek interpunct · (U+0387) are now treated as equivalent to the dot operator ( \cdot U+22c5) (#25157).
  • The minus sign ( \minus U+2212) is now treated as equivalent to the hyphen-minus sign - (U+002d) (#40948).

I’m not sure that new syntax will be hugely popular, but I would hate to be staring at code that looks identical (for dot, or very similar for minus), working in current Julia, and be confused why not working the same in 1.6 LTS. Maybe the error messages will be helpful and make obvious, but still I think bad for at least newbies.

Yes, I also hope 1.7 will support, as this seems like a very important package, and as far as I know the only package no longer supported by current Julia, nor LTS (i.e. 1.0.x), only supported by 1.3. I’m not up-to-peed on what broke it in 1.4, but if it can be fixed in 1.7, but not 1.6, then that’s an excellent reason for 1.7 to be LTS.

Some minor:

“Technically breaking”

You see that term occasionally, while never in NEWS. That’s not something to worry about for 1.7, but does it mean we wouldn’t want to backport to 1.6?

What I know of not mentioned in NEWS, lots of speedup in floating-point code (e.g. 50%), thanks to Oscar, and such aren’t mentioned as a constant factor. But there’s also fix for quadratic behavior regarding to memory allocations. It’s not mentioned in NEWS, I think because it’s a rare edge case. But I would want to avoid those in LTS too.

If we choose 1.7 and regret it, we could always add 1.6 as LTS (doubling backport effort), but if 1.6 is chosen then we can’t simply chose 1.7 and drop 1.6 as LTS, then it would be a short-term support version… However, the length/meaning of LTS has never been defined.

If 1.6 is chosen as LTS, what can we backport, or more importantly what CAN’t we backport from 1.7+? Anything obvious, good-to-have, feature?

Features aren’t backported. Full stop. The new manifest version is on the edge of being overly feature-y, but it fixes a major workflow issue.

15 Likes

The stream of random numbers in some cases already changed a few times:

% julia-13 -E 'using Random; Random.seed!(1); rand((1,2,3,4,5))'
2
% julia-14 -E 'using Random; Random.seed!(1); rand((1,2,3,4,5))'
3
% julia-15 -E 'using Random; Random.seed!(1); rand((1,2,3,4,5))'
2
% julia -E 'using Random; Random.seed!(1); rand((1,2,3,4,5))'  # Julia v1.6
4
% julia-master -E 'using Random; Random.seed!(1); rand((1,2,3,4,5))'
1
4 Likes

In fact, the stream of random numbers is documented to be able to change between any two versions. That’s a good thing - noone should rely on randomness (use property based testing or some other method to ensure your code is correct, rather than comparing with some “golden numbers”).

7 Likes

Since AD is so important for the future of Julia, I would strongly support picking whatever version for LTS that will enable the Diffractor-and-cousins. If those will require features in 1.7, then I think it is better to wait.

7 Likes

The better question is how important AD is for the risk-averse users who prefer the stability of an LTS release over new features in new minor releases. I’m kinda doubtful that cutting edge AD technology is in focus for that user group.

4 Likes

Maybe you are right, but is it worth waiting for the 1.7 beta to mature for the risk-averse users? The issue may be that a whole bunch of packages start lower-bounding at 1.7 because they want AD to work, at which point the LTS isn’t especially useful.

1 Like

LTS is only for those who wants to keep stuff that is already working working, it’s certainly not for those who wants anything new, like new features.

1 Like

For sure. But if there are compelling reasons that a bunch of crucial packages will lowerbound to 1.7 because it is the only way they can be AD-compatible, then that is a problem. It might make a bunch of people stuck with old versions of packages and unable to upgrade a year from now.

…of course, if Julia 1.7 is not strictly required for the fancy replacements for Zygote, then this may be a moot point.

I persoally inted to use the revamped version of range, and am a little sad that that code will not work in the LTS version.

I’d think that at least some of the changes to range should be able to be provided to older versions via Compat.jl.

1 Like

Someone mentioned on Slack yesterday that many optimizations in Diffractor rely on changes introduced in 1.8, so again there might not be a clear cutoff point here (as Diffractor is still under heavy development and presumably will continue to be after release).

1 Like

I can’t provide the answer, but I’d like to point out that the contenders for being a LTS are currently 1.0.5, 1.6.1, and 1.7.x.

Oh dear, I really hadn’t meant to kick off a discussion like this …

After reading the v1.7 release notes I just though “wow, this looks like a big relelase”, and I was wondering if the the plans of the dev team regarding LTS had changed, and I remembered hearing that Diffractor would require v1.7 (but based on what @nilshg wrote it looks like it’s not that clear-cut). I fully support @simeonschaub’s argument that at some point one has to be pragmatic and “bite the bullet”. The current situation with v1.3 being a kind of “de-facto LTS” (due to artifact support) certainly shouldn’t be dragged out much longer, IMHO.

1 Like

The plan is still for 1.6 to be the new LTS. There are always features in new releases that someone would like to have in an LTS version. Aside from the manifest format, which is being backported to 1.6, none of these seem like show-stoppers.

The vast majority of people should be using whatever the latest Julia version is, not staying on the LTS. There seems to be some misunderstanding that the LTS is better to run for some reason, which is very much not the case. You should only be using the LTS if it’s very expensive or risky for you to go through the process of upgrading, which is not the case for 99% of users. People who need the latest and greatest in machine learning capabilities are very much not the target audience for the LTS release and should be using whatever the current stable Julia version is. Packages should support whatever earliest Julia version they can conveniently support. If that’s the LTS, great; if not, that’s also fine. For example, for many packages Julia 1.3 has been a lower bound for a long time because it introduced artifacts and multithreading. Even though that wasn’t the official LTS, it was a fine lower bound for packages to have.

In hindsight, the amount of time spent on the 1.6 release was probably a mistake and we should just do timed releases like we normally do regardless of whether something is going to be an LTS or not. And we should probably just do a timed LTS as well on a two year cadence. So whatever 1.x release is first cut in 2023 would be the next LTS.

29 Likes