If a package becomes stable at a 0.x.y release (i.e. no future breaking changes are expected) how should that be tagged?
Tagging 1.0.0 is nice because it gives some people the impression that there are no more intended breaking changes and because it allows patch releases, but it also requires all direct dependents to unnecessarily update their compt entires.
Is there an option to register 1.0.0 as non-breaking? This would allow the Pkg resolver to interpret the 0.x compact entry as additionally allowing 1.y.z releases
Maybe this is a question for CompatHelper? If CompatHelper can figure out automatically that between 0.x.y and 1.0.0 there were no commits and that the new release just signals the intention of stability it could automatically mention that in the PR?
I think the most reliable way for CompatHelper to detect that there were no breaking changes would be an “@CompatHelper 1.0.0 does not introduce any breaking changes” comment in the repository which released 1.0.0 which could be reproduced on CompatHelper PRs as a follow-up comment “1.0.0 does not introduce any breaking changes (link to comment)”
This would have the advantage of allowing folks to declare non-breaking status for non-breaking 1.0 releases that predate this feature.
There is a mechanism for communicating release notes at registration time: https://github.com/JuliaRegistries/Registrator.jl#release-notes. Those show up in the PRs to General. Maybe CompatHelper could be taught to find those and put them in the compat-bumping PRs. So then the release notes could say “No breaking changes, …” and folks can see that in the PRs.
Well, I’d guess that if the package is stable then imposing a one-time change for dependents to update their compat bounds isn’t such a burden. This is one of the joys of maintaining a package. CompatHelper makes it pretty easy. I do not think that having CompatHelper do this automatically would work well nor is it worth the trouble.
(dependents refers to direct dependents and downloads refers to downloads within the last year excluding ci, error responses, and multiple requests from the same IP on the same day)
This is what I’d recommend and what I’ve done myself with the handful of packages I maintain: bite the bullet and go to 1.0 when it seems ready and adjust anything that has compat bounds that include the last version to also include 1.0.
StatsBase and Distributions both have some tech debt (they are old packages and you can clearly see that parts of the code are from the Julia 0.2 days!) and planned breaking improvements, but we’re very slow to do a new breaking release because they have a huge number of direct and indirect dependencies.
StatsBase in particular is a tricky case because one of the open design decisions is how to merge it with the Statistics stdlib, only the big plan now is currently to move Statistics back out of being tied to Julia releases. @nalimilan has been leading this effort and can probably say more.
On the other hand, @oxinabox has proposed somewhere that the first “true” release (i.e. non beta) should just be 1.0 because it’s not like we’ll run out of version numbers. Then we have that fine-grained resolution of “bugfix vs. feature vs. breaking” release right from the beginning. The expected mid-term API stability is then communicated via documentation.
As said, and for interest Invenia has made it a policy of not doing 0.x.y releases any more.
All new project are created at 1.0.0
and we are closing out all out internal packages to just jump straight to 1.0.0 ASAP (i thought we had gotten all of them ages ago but found some more that were not currently in prod that needed the bump)
We are being a bit slower at pushing to 1.0.0 for our open source stuff as that has more impact on others.
So are mostly trying to make the next breaking release be 1.0.0
There has been some discussion from @kristoffer.carlsson IIRC about making a tool to use with the registry that retroactively expands the compat of packages in the registry for a nonchanging 1.0.0 release to include it, and a thing in RegistryCI that detects if a new release of a package undoes it (since if the Registry change expands the bounds, but the Project.toml is not changed then it is likely that package authors might undo it with their next release.)
However I believe such a tool was just speculated upon, i don’t think anyone has done any work on it.
When a downstream package marks a compat entry of 0.3 they indicate that they are happy with version 0.3.0 and any future versions that their dependant declares are non-breaking. It seems fair, then, to add a flag in the registry that indicates 1.0 is nonbreaking which would not require making changes (automated or manual) for direct dependants to be compatible with 1.0. After all, dependents have already asked to receive all nonbreaking changes automatically.
This feels less invasive to me than retroactively editing compat entries, and fairly harmless because folks who ignore breaking = false flags will still get Symver compatible version numbers. How do folks feel about this proposal?
This could get confusing from a user point of view, if you ask for 0.3 and get 1.0 instead - and the reason for it is hidden away in a registry somewhere.
Actually this proposal would be a breaking change, as it is documented in Pkg that a 0.3 compat bound installs anything in the interval [0.3, 0.4).
IMO just release a non-breaking 1.0 in this scenario. If you’re being a very good citizen, you can make PRs on downstream packages to update the compat bounds.
It’s a little unclear what that flag should affect. The only place where semantic versioning is assumed is in the meanings of semver ranges in the compat specifications in project files. So would having breaking = false in the version for 1.0.0 mean that a compat entry of X = "0.3" would include 1.0.0 if 0.3 happens to be the highest 0.x release that has versions? How is that 0.3 derived? What if a 0.4 release is subsequently added? Does the breaking = false then have to be assumed to apply to 0.4 instead of 0.3 or do we need to know some history? I think in order for this to be workable, it would have to explicitly encode what lower versions it’s API-compatible with. And it would still have the effect that compat entries could no longer be locally decoded—they could only be decoded along with the full content of the versions file. It seems far better to me to just update the compat bounds in the registry. I don’t really get why that’s such a big problem, although perhaps I’m missing something. It is unfortunate when the registry compat for a version, which can be modified, gets out of sync with the project file of that version, which cannot be modified, but this already happens for good reason sometimes, so we may as well take advantage of the ability, imo.
The one issue with that is that since packages’ compat bounds in their Project file is not updated, as soon as someone registers a new version of their package, their compat bounds will not include the 1.0 version of the dependency. So in this case you would have a bunch of versions supporting 1.0 of a dependency (since they have had their compat updated in the registry) but the latest version would only support e.g. 0.8 since that is what is still in the Project file.
So one needs to either send out PRs to all the packages that update the compat bounds or there need to be some feedback in the registration process that the compat bounds is stale and should include 1.0.
It would be really nice to add a “push” mode of operation to CompatHelper.
Currently it operates in “pull” mode where a dependent package pulls in new versions of its dependencies.
A push mode would be for a dependency package to make PRs on all its dependents to upgrade their compat bounds. Then you could run this whenever you make a non-breaking major release (or even do it every breaking release in CI).
I’ve run into this issue a few times. It’s not a major headache to fix, but it can be time consuming and sometimes a bit surprising when you find out that you have an older version of a package than you thought, because one of your dependencies don’t have updated compat entries.
Somewhat related, another issue I’ve found is that often, packages have two or more independent feature sets. Let’s take DataStructures as an example: it exports several different collection types most of which can exist completely on their own, but they are small enough that having separate packages for each would be cumbersome. However, very often, a dependent will only use one of those feature sets, and what would be a breaking change to the others, resulting in a version bump from, say, 0.8 to 0.9, is actually non-breaking from the perspective of that dependent. If there was a way to specify which features you depend on from that package, along with the version compatibility, and if breaking changes included metadata to announce which feature sets were actually broken, it would solve both this and a number of other issues related to compat maintenance. Something similar to the Rust compat syntax of:
DataStructures = { version = "0.18", features = ["queue", "stack"] }
If a 0.19 or 1.0 release is made that introduces no breaking changes to the “queue” or “stack” features, then this would be compatible with that too.
Building on top of this, a non-breaking 1.0 release would be one in which no feature was affected. To make this change to the package manager itself non-breaking, having this forward compatibility should probably be opt-in using a special syntax or something like a features = "all". Similarly, for backwards compatibility, a package that does not include metadata about feature breakage will default to be everywhere breaking.
EDIT: The Rust “features” is a different thing, so would probably be good to pick a different name to avoid confusion.