Anyone can explain why Julia package manager works like this?
Just one example: I added the package Agents and Distributions v0.19.0 shows incompatibilities. I remove Distributions and then added it and then the incompatibility disappeared but the version of Distributions package is now v0.18.0.
It seems a behaviour very different from what happens for example in Anaconda.
Is there any rule that we must follow when these incompatibilities happen?
What’s happening here as far as I know is that Agents and Distributions are both fixed to some version, and there’s some incompatibility between Agents (or one of its dependencies) and Distributions v0.19.0.
However, when you ]rm Distributions, it get removed from your environment’s Project.toml, which
means that the version is no longer required to be v0.19.0. Then, when you re-add it, Pkg will look through the compatibility requirements of all the packages that are in your Project.toml, and determine that the highest supported version of Distributions is v0.18.0, and install that.
That being said, there seems to have been a bug with the conversion of METADATA.jl to JuliaRegistries/General, which caused package compatibility caps to be stricter than they should have been.
1 Like
Yes, there is something strange going on here with Distributions, which I guessed had something do with recent transition from REQUIRE (ie METADATA.jl) to Project.toml (General).
If you add Distributions to an empty environment and then try to add, say, KernelDensity, then you get incompatibility error. However, there is no problem adding them in the reverse order, which, as I understand it, shouldn’t make a difference, no?
1 Like
No, in your example KernelDensity.jl depends on Distributions (but not vice versa). So if you add KernelDensity to an environment (which already contains Distributions), then Distributions (in your environment) must match the requirements of KernelDensity. KernelDensity doesn’t currently have a Project.toml of its own, so it’s unclear to me where it’s requirements are set. I’m guessing it’s these lines in it’s Compat.toml in the general registry:
https://github.com/JuliaRegistries/General/blob/c0b2310f9cee4a47b49979fb07bd6eb13babf648/K/KernelDensity/Compat.toml#L23-L24
So I think if you do ]add Distributions@0.18, then
]add KernelDensity that should work.
Thanks for that! So, to clarify: Your’e saying that the order in which you add packages to an environment matters? But how can this be? Doesn’t the outcome of resolving an environment (which happens each time you add to it) depend only on the Project.toml file? And surely the order in which dependencies appear in the Project.toml file is irrelevant. What am I missing here?
Yes currently installation order matters when their are version dependencies. If you have added a package A independently (the latest version) to an environment, and later try to add package B which depends on a different version of package A, then currently the pkg manager does not auto-regrade package A. Instead the pkg manager throws a warning that the version of A you have already is not compatible with the version of B you’re trying to install.
It’s different when you add package A to an environment with B, because A does not depend on a version of B. So in this case the pkg manager gives you a version of A which is compatible with B’s Project.toml.
1 Like
So if I activate a brand new Project.toml (that I created by hand) and resolve, in what order are the packages parsed to determine possible conflicts?
When I instantiate a new environment, containing only the Project.toml below, I get Distributions 0.18 (not 0.19.2) and KernelDensity 0.5.1. The pkg manager finds the latest compatible versions.
# Project.toml
[deps]
Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
KernelDensity = "5ab0869b-81aa-558d-bb23-cbf5423bbe9b"
I remember there was some bug in General where it assumed that any package without Project.toml was incompatible with any version of a package released after the porting script was executed? That could be a reason. Would be a simple PR to KernelDensity I think to add a Project.toml and then register.
@Mattriks Or I can achieve the same by first adding Distributions, deleting the manifest (but see Pkg #1184) and then adding KernelDensity.
Thanks very much for the explanations. They dispel my confusion.
Or ]add Distributions@0.18, then
]add KernelDensity