Questions on the versioning semantics of Julia packages

I read this post that tells the rules of Julia package versioning:

But didn’t get under what circumstances will the package version string turn from a pre-1.0 form to a post-1.0 form (0.y.z1.0.0)? And does Julia’s own versioning (not only its packages’) also follow these rules?

Thanks!

Yes, Julia also follows the rules (or just very occasionally break them, if deemed a “technically breaking change” that none should discover…).

There was Julia 0.2, to 0.7 way back, usually breaking things, but stable since 1.0, i.e. you have syntax and stable API guarantee. There have been exceptions, the thread API came later and was experimental for a while. I don’t recall if that API was even broken, just not declared stable.

Basically you go to 1.0 when you believe the API is complete, you don’t know of future growing pain/need to break the API. People argue you should always start at 1.0 for open source (packages, if not languages too). Others disagree, and want to release packages early, as 0.x, “experimental” maybe, but you can also use such packages, e.g. pin versions if needed, and maybe help with development. A lot of packages are very usable, even state-of-the-art (for years even, at least back then, e.g. JuMP.jl until 2022, now in version 1.19, and DifferentialEquations.jl in 2016, now in version 7.12), and have not reached 1.0.

In packages and in Julia you are safe if you use the stable API of packages or Julia. Some packages use undocumented (not declared stable) “API” of Julia and may thus break. It’s not contradicting semver rules, basically packages shouldn’t (at least usually). Then the “fix” is to upgrade to newer version, it almost always works. I know of only 1 or 2 packages that couldn’t be supported for later Julia versions.

1 Like

Mostly this is laid out in https://semver.org

under what circumstances will the package version string turn from a pre-1.0 form to a post-1.0

When the API is “stable”. That is, when you’re no longer wildly experimenting with your code and breaking the API in every release. One common misconception (and why a lot of packages stay on 0.x far longer than they should) is that 1.0 must be breaking. I’d argue that in general, v1.0 should functionally match the last v0.x. It’s just a “stamp of approval” to indicate that it’s stable. Although it should also “clean up” any deprecated code from the v0.x series, so that can often technically be “breaking” and is fine.

And does Julia’s own versioning also follow these rules

Officially, yes, although for big projects like Julia what is considered “breaking” can be a little hand-wavy. With a programming language, pretty much any change will break someones code in some weird edge case. But Julia certainly promises not to make any large backwards-incompatible changes in the 1.x series (and would also be very averse to switching to 2.0 in the near future, if at all)

3 Likes

That’s both a misconception and not. Semantic versioning leaves everything open before 1.0, so from that point of view you are free to change how little or much you want when going to 1.0.

However, Julia’s package manager will interpret it as a breaking change with all that entails in terms of compatibility constraints, so you need to take that into account as well. If nobody depends on your package that’s a very small concern but if there are a lot of packages depending on it, directly or indirectly, the cost is higher and of questionable value just for juggling version numbers.

3 Likes

Yes, and I think Julia’s package manager’s interpretation of SemVer is idiosyncratic at best: Should You Use Upper Bound Version Constraints? -

I worry that sooner or later, the way we treat versions is going to blow up in our collective faces. But maybe not: the Julia community is a little bit more cohesive than other communities, and maybe will manage to make sure that everyone stays on top of all their dependencies (*). At the very least, I wish we’d stop treating 0.x releases as “necessarily breaking”. Almost no one actually makes 0.x releases in a disciplined matter. Most people use 0.x as “big change” and 0.x.y as “little change”, and officially SemVer is pretty much “anything goes <1.0”, which makes that fine. Plus, If I take on an experimental dependency, I’m already opting in to an unstable API. This is whole rant is actually more of a problem with General registry policies, not so much Pkg itself. I should be able to register a package with “infinitely many breaking versions” as a dependency if I choose. Maybe it’s good to require a confirmation override to make sure it doesn’t happen accidentally. I don’t even want to get into what happens if I’d prefer CalVer for some project. For some projects, it’s a much better fit, and if it weren’t for the idiosynchratic reading of SemVer that has taken hold here, it would be 100% compatible with SemVer.

(*): That’s kind of a sympton, not a cure, though: I presume by " juggling version numbers" you mean that lots of people will get yelled at by CompatHelper. Because beyond that, not having 1.0 breaking only makes it easier for downstream developers, as they can update at their leisure.

</ end rant> :wink:

2 Likes

The issues with SemVer all stem from it being a social solution to a technical problem. The package maintainer pinky-swears to “not break the API” without incrementing the major version number, but without any mechanism to define the behavior of the API, or what the difference is between a bugfix and a breaking change.

This can be turned into a technical solution, without underspecification or arguments, in the following way: require major versions to have a @testset called "API". Every test in API must stay green on any minor or patch release, tests can be added on minor or patch releases, but never removed. Each of these tests must have a unique name, for reasons I’ll get into.

What’s tested may be relied on, what is not tested may not be. The package manager could refuse to create a minor or patch release which breaks this contract, automatically. If you’re unsure what behavior is considered reliable, consult the tests. If you think something is missing, open up a PR with additional tests in the API.

For major releases, tests removed from the API get moved elsewhere and given a different macro, like @test_obsolete. There can be a deprecation process, where the tests which will break in an upcoming release get tagged as deprecated. No API test is ever renamed, changed, or removed, this is the most important contract in the entire system.

That would let packages to name the API tests they rely on in their own test suite. If they make a minor upgrade which introduces a deprecation, they’ll see it as soon as they run the test, and have a chance to apply the changes before the major version bump.

This provides an actual semantics to semantic versioning, and a universal mechanism for resolving disputes about what constitutes a breaking change. It removes cause for complaint if a bugfix changes something which a package relies on, and also gives users of the package a way to ask that some behavior be considered part of the API: write a test for it and submit a PR. If that lands, the package manager won’t allow a release in that major version number to break the behavior.

5 Likes