It’s not just not easily fixable, it’s impossible. Anyone can write a code that uses an internal that some one says “please don’t rely on this, it’s not guaranteed to be stable”. All you need is a single package in an entire manifest to rely on something that they are explicitly told not to, and then boom you have this result. And many packages don’t even do this accidentally, many packages that integrate with the compiler do this on purpose and then directly version which versions of Julia they work with.
3 Likes
Would it make more sense to slow down Julia releases after an LTS that includes most of the machinery needed for those cutting-edge AD libraries?
Well, absolute compatibility is indeed impossible because it means no changing anything. But in Julia, the contrast between advertisements like
and reality does feel like a betrayal when encountered for the first time.
In reality, one basically cannot keep code intact for a few years and hope it works with new julia versions. Recent breakages have happened in packages so popular and with so many dependents, that it was kinda impossible to avoid in any nontrivial environment.
As I said multiple times already, I’m not arguing this is fundamentally wrong! It’s just that too strong promises shouldn’t be advertised as much.
1 Like
I think you are exaggerating a bit the problem here. As a package developer that does not use internals, my packages install and run just fine on any version of Julia, and with any version of the packages. Some of the packages rely on LoopVectorization, but that dependency gets sorted out correctly because it (LV) has the correct lower bounds for Julia. Yes, I cannot use a manifest file that relies on a newer LV version and my package because the newer LV is only compatible with newer Julia, but no Julia update broke my packages ever.
So, while these packages that rely on internals (or any new features) keep their Julia bounds correct, we other package developers should not experience these compatibility issues in everyday use, and that’s why I do not see them ever.
ps: Of course your experience may differ depending on the toolkit you rely on. But I think that the add RandomPackage is pretty robust in Julia, in any version. I would bet that one needs to do cherry picking to find a package that stopped working in that sense in a new Julia version, that worked previously.
5 Likes
Let’s take one concrete recent example out of many. All environments that include StatsBase with versions pinned a few months earlier or before, stopped working in Julia 1.9. How I, as a user, know that code in such fundamental packages isn’t supposed to stay compatible, and I need at least regular package upgrades so that my program works on newer julias? Especially given the widespread advertising of “everything continues to work for Julia 1.x, users don’t need to do anything”.
Maybe it’s cherry picking, but these cherries are among the most popular packages (:
The cherry picking I think is in this phrase. If you are pinning, then yes. That I agree. If you aiming to exact reproducibility, you need to keep the Julia version (and it is possible that this should be better advertised). But I would guess that you can install the same packages (without pinning) in the newer Julia versions, for the same environment.
1 Like
Well, “update packages and most of them (those updated by the authors) will work on new Julia version” doesn’t say basically anything on Julias backwards compatibility.
Yes, one can upgrade all packages (and effectively has to!), but this involves dealing with package updates — not only Julia updates: new bugs, deprecations, regressions, etc.
3 Likes
Yes, you’re right. I think I always saw that claim in the sense as not having to worry myself as a package developer that my package would suddenly stop working in a new Julia version because of a breaking syntax or feature change.
I think a more precise phrasing should be
If your code only depends on public methods from
Baseandstdliband it works now it will work until Julia 2.X.
As all past discussions about this revealed, all bets are off once you add any other dependencies.
We’ve talked about all this before in a previous thread, but to be clear, lower bounds on dependencies is not the only issue with reproducibility in the Julia ecosystem. The other issue is this:
Any package that relies on internal functions from Base or stdlib should place an upper bound on the Julia version in the compat section of their Project.toml, but I’ve never encountered a Julia package relying on Base internals that does this.
To take a specific example, DataFrames.jl extends Base.dotview and Base.dotgetproperty, which are not part of the public API of Base Julia. But DataFrames does not place an upper bound on the Julia version. Version 1.6.1 of DataFrames works on Julia 1.9. Suppose I have a package ReallyCoolPackage.jl with the following compat section in my Project.toml file:
[compat]
julia = "1.6"
DataFrames = "=1.6.1"
Now suppose Julia 1.10 comes out and Base.dotview and Base.dotgetproperty have been removed from Base. If I upgrade to Julia 1.10 and I run ] instantiate --project in my package directory, the dependency resolver will happily install my environment. But when I run the code in my package, it will be broken if it uses any part of the DataFrames API that internally calls Base.dotview or Base.dotgetproperty.
The point here is that an environment that is nominally feasible is actually infeasible because a dependency that uses Base internals did not upper bound the Julia version.
I’m not trying to pick on DataFrames, that’s just the example of internals usage that I’m most familiar with. Unfortunately, I think the practice of using Base internals without upper bounding the Julia version is actually pretty common in the Julia ecosystem.
10 Likes
I don’t think what you are describing is something that has to do with reproducibility. For that, I would say you should really run with the same Julia version. You seem to more describe possible breakage when updating Julia versions which is in my opinion a different matter.
1 Like
Yeah, I guess “reproducibility” is not the right word here. The issue is “environments that are installable and should work based on the declared compat bounds, but which don’t actually work because the compat bounds are wrong.” I’m not sure what the word for that is.
Of course that sentence applies to both lower and upper bounds, whereas I was focusing on the lack of appropriate upper bounds in some packages.
For what is worth, are there good solutions for these kind of problems in other languages or ecosystems?
I used to do so for LocalRegistry, although technically it relied on Pkg internals rather than Base internals. The major drawback of this is that your package now won’t be installable in unreleased Julia versions (think master or nightly builds) which you haven’t yet bumped your compat bound for, because you don’t know whether it will stay compatible with those versions.
Btw I dropped the upper bound recently because I removed the dependencies on the internals, except in the tests.
2 Likes
Is there any way to programmatically find out whether some function is in official API or internal?
4 Likes
If there were such a way (and doesn’t look like it comes soon), then it would be possible to automatically request an upper limit for Julia version (or for package dependencies) during the package registration.
1 Like
Well, a partial solution is work-in-progress: https://github.com/JuliaLang/julia/pull/50105
1 Like
(Apologies if anyone mentioned this already, but I didn’t see it.)
My understanding, which has been stated many times in various places (IIRC), is that the “if it works now it will work until Julia 2.x” statement is only for the Julia language itself, not the packages. So, if I write 10000 lines of code with no package dependencies for Julia 1.9, then it should work in 1.7, 1.8, 1.10, etc.
As a comparison, over the course of Python 3.x, different ways to to string interpolation were added as features, including f-strings in 3.6. So if I wrote a package for Python 3.11 with the following lines
x = 1
print(f"x = {x}")
Then my package would break for all users still on Python 3.5 or earlier, yet no one (anyone here?) considers 3.5 → 3.6 a “breaking” Python release. So one question is, given a completely analogous situation, why is does the bar seem so much higher for Julia than other languages?
5 Likes
and even then, only for exported or publicly-documented behavior
I think your analogy might have the comparison backwards. shouldn’t you be looking for examples of user code that work on 3.5 and then break on 3.6+ ?
2 Likes