Dependency problem with FIllArrays

https://github.com/JuliaLang/Pkg.jl/pull/1163

Ok, so tag by just bumping by a patch then? The whole point of bumping a minor version in < 1.0 is to indicate that it is breaking release and that packages that depend on it shouldnā€™t automatically use that version without checking if it is compatible with the breaking changes.

I tagged FillArrays.jl as v0.6 because transpose/adjoint/vec now return an immutable Fill. This is ā€œbreakingā€ because code that assumed these functions are mutable would be broken. But it is very unlikely that any code was actually broken.

Should this have been a patch then?

It sounds like a breaking change so packages that depend on the package should thus not automatically be compatible with this version but instead verify that that they are either not affected by the breaking change, or update their package to be compatible with the new release. When that is done, they release a new version, adding compatibility with the breaking release of the dependency and thus Pkg.update() will not break anyoneā€™s code. I donā€™t see any other way of doing it.

Assuming that breaking changes do not apply to you and then changing the registry in hindsight when you start getting bug reports about the package not working anymore seems like a much worse way of doing it. Pkg.update() is then very likely to just flat out break users previously working setup of packages.

1 Like

But this has been and still is up to the package developer: we can still do FillArrays = "ā‰„ 0.5.0" and automatically use new versions with breaking changes.

The issue is that users have said they want this behaviour by writing

FillArrays 0.5

in REQUIRE, which has been reinterpreted as FillArrays 0.5 0.6 without the package developers input.

Worrying about users setup breaking is premature before a package is tagged as 1.x. It seems like you are trying to reinvent 0.1.x as the new 1.x.

1 Like

Yes, you can but for now we have a correct start. Upper bounds was too annoying in the REQUIRE format for anyone to use it. Now it is the easiest to use. For the pain this have caused, start going through Pull requests Ā· JuliaLang/METADATA.jl Ā· GitHub. Accepting stuff like that is very much up to the maintainers of the registry.

How Pkg interprets semver pre 1.0 is documented and is consistent with how Julia itself has been developed and how many other package managers do it (like Cargo).

1 Like

This is just to make sure I understand (I donā€™t have a strong opinion either way but would like to understand what will happen). In a few packages I have put ā€œrelaxedā€ compatibility requirements (as a practical decision, I accepted that in 0.x versions there would be occasional breakage and planned to be stricter when packages get closer to 1.0 releases). Can I assume that versions were ā€œstrictifiedā€ according to semver in the registries and will stay strict until I tag a new release, in which case they will respect what I wrote in Project.toml?

As far as I know, that sounds correct.

1 Like

Ok I get it: you are assuming most packages didnā€™t put upper bounds not intentionally but because it was too hard. Fair enough. I suppose this will sort itself out over time.

1 Like

Can you clarify why you prefer adding bounds manually afterwards instead of having them be correct from the start? e.g. https://github.com/JuliaLang/METADATA.jl/pull/19766, https://github.com/JuliaLang/METADATA.jl/pull/19308, https://github.com/JuliaLang/METADATA.jl/pull/19120, https://github.com/JuliaLang/METADATA.jl/pull/18841, https://github.com/JuliaLang/METADATA.jl/pull/15386, https://github.com/JuliaLang/METADATA.jl/pull/15210, https://github.com/JuliaLang/METADATA.jl/pull/13524, https://github.com/JuliaLang/METADATA.jl/pull/7584 etc. From these it looks like the current behaviour is what you really want? But maybe I am missing something?

3 Likes

Good point, perhaps Iā€™ll start putting upper bounds in. Though it depends on the package: FillArrays.jl is unlikely to change much in between versions.

Upper bounds are ā€œbuilt-inā€ in the new system in the sense that the next breaking (as defined in the docs) is not allowed. Therefore, something

[compat]
Package = "0.1"

means compatibility with the 0.1-series of releases, but not 0.2 since that is considered breaking.

2 Likes

And ā€œ0.1.1ā€ means ā€œ0.1.1 <= VERSION < 0.2ā€?

Yes, see 6. Compatibility Ā· Pkg.jl

By the way, if you want to encourage this behaviour, perhaps @StefanKarpinskiā€™s gen_project.jl script should be changed as well to not produce ā‰„ 0.x.y by default?

The script conservatively translates your REQUIRE bounds as literally as possible. Otherwise Iā€™m sure you can see that someone would be complaining that their package which previously installed just fine now wonā€™t install because the generated compat section is much more restrictive than the REQUIRE file was.

It seems like some degree of human judgement about compat bounds is required here which is why Iā€™m not very enthusiastic about automatic translation of the old bounds to the new onesā€”I think thereā€™s a hard limit on how well it can be done completely automatically and package authors will need to start thinking about and understanding semantic versioning.

1 Like

That argument also applies to the original translation that caused this threadā€¦

Yes, Iā€™m damned if I do and damned if I donā€™t, which is why Iā€™m mostly ignoring complaints.

4 Likes

Fair enough. Same goes for git knowledge. I see it as an opportunity to learn and improve in something I suck at.

Please donā€™t be disheartened by complaints. The discussions, coding, and conceptual design devoted to the new Pkg system by you and others is highly appreciated.

4 Likes