LoopVectorization in v0.12.110 had ArrayInterface compat entries for 3.1.32, 3.2.1, and 5.0.1. Then v0.12.111 dropped those in favor of v6.0.1.
Even though it’s a dependency and thus doesn’t break the API of LoopVectorization, it does create problems for any other packages that depend on ArrayInterface.
One reason I think it should be considered a breaking change is that, if for whatever reason a package is stuck with v0.12.110 due to compat bounds from other packages, there’s no way to deliver fixes for any bugs in that version independently.
Not quite the topic but, what hasn’t updated to ArrayInterface 6 yet? It’s been half a year and the change is usually 1-5 lines of code for this update. I’d be happy to go fix that if I know what still bounds it.
I think the issue is more that dependencies are interacting, leading to downgrades in those packages, that then require an older ArrayInterface version.
Considering changes in compat bounds as breaking changes would mean that the major version should be bumped, which is quite radical and impractical. But indeed it could be a good idea to only change compat bounds in minor (a.k.a feature) releases (as opposed to patch releases), so that bugfixes can still be backported to people who need old versions of dependencies. This is the policy that is recommended when dropping support for Julia version, for the same reason.
Now the situation for packages that have not reached 1.0 (LoopVectorization.jl) is more tricky as there are no digits left to distinguish patch releases from minor releases. I think the general sentiment is that such packages should tag 1.0 to avoid this problem.
I agree in general that backporting fixes is useful and have done so myself several times in other packages.
For the specific example given, updating to ArrayInterface 6 is hopefully easier and a better use of time than backporting fixes to LV.
It should improve latency, and allow for compatibility with the rest of the ecosystem that has moved on.
Of course, I understand that the problem may be that you’re faced with an LV bug while also stuck on old versions of the rest of the ecosystem for other reasons…
It can absolutely be painful when the package ecosystem is split on different sides of a breaking change, but trying to combat that by forcing even more breaking changes sounds counterproductive.
The breaking change in this case would be dropping support for ArrayInterface < v6. Incrementing semver to indicate that breaking change isn’t “more breaking”, it would just be recording the fact that a break occurred, and allow packages to deal with that break however they wish.
Indeed, hopefully it is less effort - but as it stands, it can be very difficult to determine which specific package is the main cause when multiple shared dependency downgrades are happening at once. And then once identified, there’s getting the various repository owners to update the compat bounds in a timely manner. Even some of those with the Compat Helper bot I’ve noticed lingering, with plaintive bump requests by hopeful users.
It wouldn’t be “more breaking” but it would be more packages with breaking changes, and another couple rounds of Compat Helper PRs that may or may not be merged in a timely manner.
If I were to release LoopVectorization 0.13, a lot of people who neither know nor care about ArrayInterface are suddenly going to have to update compat bounds on packages.
Post 1.0, that wouldn’t be a problem.
Maybe I should release LoopVectorization 1.0.
Future development will go into a separate package with a separate git history, so LoopVectorization.jl is as mature as it is going to get. It’s also currently at 0.12.128; it has tried hard not to break anything.
As an exercise, I wondered “How much effort would actually be needed to maintain backward compatibility”?
Here’s what I found:
First, it’s not possible to express that “uninstalled” is a valid state for a direct dependency. This would be needed to support a state where either the monolithic ArrayInterface early versions could be used OR slimmer ArrayInterface v6 + individual support packages. Pkg could have better support for merging and splitting packages across versions.
Also could use a way to provide conditional code based on dependency version - judicious try/catch can work, but not ideal. Though maybe I’m ignorant on this ability if it already exists. (I am aware of how to do this for Julia versions).
Apart from the mapping between ArrayInterface packages, the only roadblock I identified was that a patch update of Static.jl in 0.5.x changed the name of a function from _get_tuple to field_types. Patching that and expanding compat bounds was enough to get LoopVectorization to precompile successfully and to pass all its tests using ArrayInterface v3.1.33.
(I was able to get Static 0.6.6 to work with both the LoopVectorization current version and ArrayInterface v3.1.33; 0.7 introduced deeper problems.)
So… not picking on LoopVectorization or ArrayInterface specifically (and I know handling package splitting/merging is likely not a trivial matter)… but maybe there’s more that could be done, at least theoretically, to maintain broadest compatibility, with minimal effort?
Thinking more broadly, beyond this specific example -
Another suggestion might be to prevent auto-merge for the registry when compat bounds are restricted in a non-breaking release, at least to ask the submitter to verify that it’s truly necessary and there isn’t an easy compatibility patch they could apply instead.