The issues with SemVer all stem from it being a social solution to a technical problem. The package maintainer pinky-swears to “not break the API” without incrementing the major version number, but without any mechanism to define the behavior of the API, or what the difference is between a bugfix and a breaking change.
This can be turned into a technical solution, without underspecification or arguments, in the following way: require major versions to have a @testset
called "API"
. Every test in API must stay green on any minor or patch release, tests can be added on minor or patch releases, but never removed. Each of these tests must have a unique name, for reasons I’ll get into.
What’s tested may be relied on, what is not tested may not be. The package manager could refuse to create a minor or patch release which breaks this contract, automatically. If you’re unsure what behavior is considered reliable, consult the tests. If you think something is missing, open up a PR with additional tests in the API.
For major releases, tests removed from the API get moved elsewhere and given a different macro, like @test_obsolete
. There can be a deprecation process, where the tests which will break in an upcoming release get tagged as deprecated. No API test is ever renamed, changed, or removed, this is the most important contract in the entire system.
That would let packages to name the API tests they rely on in their own test suite. If they make a minor upgrade which introduces a deprecation, they’ll see it as soon as they run the test, and have a chance to apply the changes before the major version bump.
This provides an actual semantics to semantic versioning, and a universal mechanism for resolving disputes about what constitutes a breaking change. It removes cause for complaint if a bugfix changes something which a package relies on, and also gives users of the package a way to ask that some behavior be considered part of the API: write a test for it and submit a PR. If that lands, the package manager won’t allow a release in that major version number to break the behavior.