Dependency problem with FIllArrays

From the docs (https://julialang.github.io/Pkg.jl/v1/compatibility/):

While the semver specification says that all versions with a major version of 0 are incompatible with each other, we have made that choice that a version given as 0.a.b is considered compatible with 0.a.c if a != 0 and c >= b .

But it is? You can have

[compat]
Package = ">0.1"

to support anything higher than 0.1 (but note that that is not true, you can not guarantee that you are compatible with versions that is not even released yet). Even worse, you can leave out the compat completely and be “compatible” with everything.

2 Likes

I know, I’m referring to the upper bounds added “against our will” to my packages (and the OPs packages) on 11 April during the transition. I’m arguing these should be undone.

Those were added for good reasons (there was a long thread about it) and have been in place for ages. You have also been able to remove them for just as long in various ways. Nothing has changed now. It’s unclear to me why this is suddenly a big problem.

2 Likes

I wouldn’t call 6 days “ages”: tracking the history they were added on 11 April. It’s a big deal because it’s unintentionally breaking downstream packages like the OPs, and also forcing me to immediately tag new versions of about 10 packages for anything to work (the package manager just gives up if no compatible version is found) …

1 Like

I understand the default is the “tilde specifier” for [compat] of Project.toml. But isn’t the translation of REQUIRE the problem? The Julia 0.6 documentation says

For example, the line:

Distributions 0.1

is satisfied by any version of Distributions greater than or equal to 0.1.0.

This translates to Distributions = ">= 0.1" in Project.toml, right? If so, why a new upper bound is introduced in the conversion of METADATA to General?:

3 Likes

It didn’t matter since they were adjusted on each new sync, so the bounds were effectively >= 0.1.

Thanks, that explains why it has been working nicely. But I suppose it’s started to matter now that there no more METADAT-to-General conversion? That is to say, shouldn’t the information in General originated from REQUIRE files keep respecting the specification in the Julia 0.6 documentation?

1 Like

Ok, so tag by just bumping by a patch then? The whole point of bumping a minor version in < 1.0 is to indicate that it is breaking release and that packages that depend on it shouldn’t automatically use that version without checking if it is compatible with the breaking changes.

I tagged FillArrays.jl as v0.6 because transpose/adjoint/vec now return an immutable Fill. This is “breaking” because code that assumed these functions are mutable would be broken. But it is very unlikely that any code was actually broken.

Should this have been a patch then?

It sounds like a breaking change so packages that depend on the package should thus not automatically be compatible with this version but instead verify that that they are either not affected by the breaking change, or update their package to be compatible with the new release. When that is done, they release a new version, adding compatibility with the breaking release of the dependency and thus Pkg.update() will not break anyone’s code. I don’t see any other way of doing it.

Assuming that breaking changes do not apply to you and then changing the registry in hindsight when you start getting bug reports about the package not working anymore seems like a much worse way of doing it. Pkg.update() is then very likely to just flat out break users previously working setup of packages.

1 Like

But this has been and still is up to the package developer: we can still do FillArrays = "≥ 0.5.0" and automatically use new versions with breaking changes.

The issue is that users have said they want this behaviour by writing

FillArrays 0.5

in REQUIRE, which has been reinterpreted as FillArrays 0.5 0.6 without the package developers input.

Worrying about users setup breaking is premature before a package is tagged as 1.x. It seems like you are trying to reinvent 0.1.x as the new 1.x.

1 Like

Yes, you can but for now we have a correct start. Upper bounds was too annoying in the REQUIRE format for anyone to use it. Now it is the easiest to use. For the pain this have caused, start going through https://github.com/JuliaLang/METADATA.jl/pulls?utf8=✓&q=is%3Apr+upper+bound+is%3Aclosed+. Accepting stuff like that is very much up to the maintainers of the registry.

How Pkg interprets semver pre 1.0 is documented and is consistent with how Julia itself has been developed and how many other package managers do it (like Cargo).

1 Like

This is just to make sure I understand (I don’t have a strong opinion either way but would like to understand what will happen). In a few packages I have put “relaxed” compatibility requirements (as a practical decision, I accepted that in 0.x versions there would be occasional breakage and planned to be stricter when packages get closer to 1.0 releases). Can I assume that versions were “strictified” according to semver in the registries and will stay strict until I tag a new release, in which case they will respect what I wrote in Project.toml?

As far as I know, that sounds correct.

1 Like

Ok I get it: you are assuming most packages didn’t put upper bounds not intentionally but because it was too hard. Fair enough. I suppose this will sort itself out over time.

1 Like

Can you clarify why you prefer adding bounds manually afterwards instead of having them be correct from the start? e.g. https://github.com/JuliaLang/METADATA.jl/pull/19766, https://github.com/JuliaLang/METADATA.jl/pull/19308, https://github.com/JuliaLang/METADATA.jl/pull/19120, https://github.com/JuliaLang/METADATA.jl/pull/18841, https://github.com/JuliaLang/METADATA.jl/pull/15386, https://github.com/JuliaLang/METADATA.jl/pull/15210, https://github.com/JuliaLang/METADATA.jl/pull/13524, https://github.com/JuliaLang/METADATA.jl/pull/7584 etc. From these it looks like the current behaviour is what you really want? But maybe I am missing something?

3 Likes

Good point, perhaps I’ll start putting upper bounds in. Though it depends on the package: FillArrays.jl is unlikely to change much in between versions.

Upper bounds are “built-in” in the new system in the sense that the next breaking (as defined in the docs) is not allowed. Therefore, something

[compat]
Package = "0.1"

means compatibility with the 0.1-series of releases, but not 0.2 since that is considered breaking.

2 Likes

And “0.1.1” means “0.1.1 <= VERSION < 0.2”?