setup-julia@v2 will install Julia 1.10.0, because that’s the absolute lowest Julia version that is compatible with the Julia compat entry in your Project.toml file.
However, I (and a few other folks) find the current behavior a little surprising. I would instead like[1]min to install Julia 1.10.10 (or whatever the latest 1.10.x patch is). In other words, I’d like min to install the minimum major/minor, but the latest patch.
I think that this is probably technically a breaking change, so my plan is to make this change in setup-julia@v3.
(To be clear, if you’re currently using setup-julia@v2, your CI jobs won’t suddenly start breaking. You’ll need to upgrade to setup-julia@v3 to get the new behavior.)
What do folks think of this proposed change? Is there anyone that specifically wants the old behavior?
I think that the current behavior is a bug, and the change is not breaking, it is a bugfix, and should just be released. I cannot imagine why anyone would want 1.x.0 instead of 1.x.y (with the latest y) and have x calculated from the Project.toml.
I don’t think anyone relies on the current behavior, for most people the difference is irrelevant, and for the rest it is a nuisance. I was actually surprised by this behavior, tracking down a resolver bug that was fixed and backported ages ago, but is still present in 1.10.0. Does anyone really need bug-compatibility in CI, and calculate the version from the Project.toml? Those two goals seem incompatible.
I don’t get this at all. The documentation of min is “will install the earliest supported version of Julia compatible with the project”. That’s pretty unambiguously 1.10.0. If I declare my package compatible with 1.10.0 and it doesn’t work on 1.10.0, only on 1.10.10, then I’d want to know about that so I can adjust my compat bound. That’s presumably the purpose of min.
How? It matches exactly the documentation. The new behavior does not. I would definitely report the new behavior as a bug if I came across it without being aware of this discussion.
I was actually surprised by this behavior, tracking down a resolver bug that was fixed and backported ages ago, but is still present in 1.10.0
Sure, but that means your package isn’t actually compatible with 1.10.0, right? And, you shouldn’t declare that it is. Annoying, but didn’t the check do you a favor?
But more importantly: I think the more dependents a project has, the more conservative it should be about breaking changes. Basically every Julia package’s CI depends on setup-julia. So, if you release a breaking change, I’ll have to deal with a dozen or more PRs from dependeabot. I’d say avoid making a breaking release more than once every 10 years.
Seems like the only reasonable option to me, and also semantically correct.
Now, if everyone thinks I’m crazy and that nobody ever wants to test against what is actually the lowest compat, or that declaring julia = "1.10" in my package’s compat really means “the latest 1.10.x”, in practice, then probably declare this a “bugfix” and release it as such. I prefer slightly questionable interpretations of “what is a ‘bug’” to the churn of having to deal with a breaking release of a common dependency; see [docs] No documented breaking changes in v7 · Issue #776 · actions/upload-artifact · GitHub for the source of my current crankiness on this topic
No, from my perspective this is just a bug. If I ever want to run CI on an earlier Julia version that has a bug that has been fixed since, I will ask for it explicitly.
As a package author, I just want to assume that for each Julia minor version, the user has the latest patch installed. This may not happen, but [compat] is not where I want to enforce it, since Julia bugs are unlikely to affect just a single package.
The alternative is to go crazy. Eg consider a (hypothetical) generic bugfix backported from 1.13 to 1.12.9, 1.11.10, and 1.10.14 (all those numbers are made up). Assume it fixes something that has nothing to do with my package, eg a generic resolver bug that can just show up anywhere under the right conditions. Should I maintain a complex [compat] entry for this?
The purpose of min in this action is to make life simple for package authors.
I don’t think you are, but I wonder if this does practically affect you, or if you are just disagreeing on principle.
What would be useful to see is a comment from someone who actually wants to test on 1.x.0. Then we could ask why and proceed from there.
At least for my packages > 1.0, I do make a point of testing against the lowest compat bounds
I think it’s important to do that. Now, that section actually contains a bug. I wasn’t aware of the min option for setup-julia, so I used 1.6 without understanding the documented behavior
'1.0' is a version range that will match the highest available Julia version that starts with 1.0
That should have been 1.6.0, or, in fact, min with the current behavior. So I think min is extremely useful, but only if it actually works the way it does now and is equivalent to 1.6.0.
Either your package (the test suite, that is) runs on the set of minimum versions you declare, or it doesn’t. If it doesn’t, then yes, you should modify your declared compatibilities. Maybe I don’t understand how this generic resolver bug affected the package.
I could see an unfortunate situation if there’s an issue with a test-only dependency. That is, the package works under the declared compat bounds, but you can’t run the test suite because of a resolver bug. In that case, that’s unfortunate, but then you probably will have to not use min.
But generally, I would hope that “complex” [compat] spec would still be rare. But yeah, if your package only works with 1.13.0 and upwards, and with the bugfix that was packported to 1.12.9, 1.11.10, and 1.10.14, but it doesn’t work with 1.12.8, 1.11.9, and 1.10.13, then the only correct options are a complex [compat] spec, or making your life simple and requiring 1.13.
No, I don’t think that at all! The (documented) purpose of min is to test against the minimum declared compatible version of Julia. If you want “simple”, then I would say use the manual 1.x, like I was doing in my example (mistakenly, because I didn’t know about min). Or use the newly proposed min-minor, which does something “simple” and probably reasonable, but definitely doesn’t test the minimum compatible version.
So yeah, I’m definitely someone who actually wants to test on 1.x.0, since I think minimum-compat-bound testing is extremely important and every stable package should do it. Moreso for the package dependencies, but I don’t see why Julia itself should be exempt from a strict test. So this isn’t just disagreeing on principle
From my perspective, the main purpose of declaring compatibility with a specific version of Julia is making sure that I have the API that I need — eg that I don’t try to use (; a, b, c) = nt in Julia 1.6.
I think that tracking patch versions to see what bugs were fixed is impractical, especially if you consider all dependencies. I rely on the resolver to make sure to install the latest version. Ideally that would apply to julia, but in practice this actions script takes care of that.
There’s a fairly large difference between a package not working with 1.10.0 because of a bug in 1.10.0 and not working with 1.10.0 because the package is API incompatible.
Those who want to verify API compatibility are generally best served by testing with 1.10.max, under the assumption that’s the 1.10.x release with fewest bugs.
It’s not unreasonable to want to test if your package breaks due to Julia bugs but I’d question whether it’s sufficient to test 1.10.0 in that case. There might be a bug introduced in 1.10.4, breaking your package, that’s fixed again in 1.10.5 and then you would want to know that as well, right?
That absolutely makes sense, and I think there’s two distinct and valid use cases here: making sure that a package remains generally compatible with some minimum of Julia (API compatibility), and then the strict compat testing.
I think that the wording of the existing documentation of min points to the strict compat testing as its use case, but people were using it for the looser API-compatibility testing. Hence the confusion and frustration, and the desire to change it.
I would still say that I’m personally interested in the strict compat-testing, so I would find min with its current semantics very useful (even if I just found out about it today). So changing both the documentation and the behavior of min seems like something that’s both iffy from a semver perspective (you’d have to argue that neither the docstring nor the implemented behavior reflected the actual intent), and removes the ability to do strict compat-bound testing without spelling out the version by hand.
So the addition of min-minor still seems like the most appropriate solution to me. It’s definitely a useful feature. Unfortunately, that means you and anyone else who has been using min with the intent of loose API-compatibility checks would have to change their CI to min-minor. I can live with changing min, too, as well as with a major release to get around the potential “iffiness”; although I really don’t like the churn of having to deal with breaking releases.
Fair point, and there’s also the question of what one declares vs what one can reasonably test. I’m not fond of going too crazy with CI test matrices. While GitHub is generous with free CI, we’re still collectively burning a lot of compute for tests that don’t give the most bang for the buck. Thus, I usually restrict to testing only the minimum and latest version of the dependencies, and not the versions in between (or all possible combinations). For the minimum required Julia, I think both testing against 1.x.0 and 1.x.y can be effective in that regard, and may be a matter of taste. If you then get a bug report that says "your package doesn’t actually work with 1.x.0, you can (and probably should) still adjust the declared compat bounds, independently of what is tested on CI.
To give a bit more context: We already need to make a breaking release of setup-julia. Specifically, we need to upgrade from Node 20 to Node 24, which is a breaking change[1]. So, we have to make a setup-julia@v3 anyway.
I suspect the 1.x.y for min is the more popular use case, so I agree: if you have to make a breaking release anyway, change the behavior for min and add a new name for the old behavior. I’d probably choose min-strict, but whatever people like should be fine.
Bike shedding now but you could have min-patch for the current behaviour, min-minor for the proposed behaviour and min as an alias for min-minor. The longer names are more explicit.
