I think Stats.jl is in the blind spot; it is causing some test failure due to incompatible versions as of today. Work around automatic Pkg capping by mohamed82008 · Pull Request #7 · JuliaStats/StatsKit.jl · GitHub
This is not exactly a run-of-the-mill situation: we are in the gradual process of swapping out the whole package management infrastructure of the language while people are using it. It feels a bit like this:
Or, if you prefer a more mechanical metaphor, changing the engine on a plane while it’s flying. The fact that it’s gone so smoothly thus far is, frankly, a small miracle. There was a minor hiccup that prevented you from upgrading your packages for what, a couple of hours? My original post also contained a work-around if you found yourself in that situation and wanted to get around it.
Perhaps there was some other better way of getting accurate version caps for all the packages and versions of packages in the ecosystem, but if there is, I have yet to hear it. The process we followed (discussed and planned here) was:
- come up with a good heuristic for which package versions should be capped
- use PkgEval to come up with known exceptions to the heuristic
- test it in advance as much as possible to try to make sure things are working
- apply the caps
- fix cases that were not quite right as people report them
I think it went fairly well. Adjusting the caps from 0.7 to 1.0 was a good idea that didn’t come up before we tried it, but was done within about an hour. What would you have done differently?
13 Likes
To answer my own question, if I were to do it again, I would communicate that we were doing this better and further in advance.
7 Likes
It seems that FastClosures.jl should also not be capped:
ERROR: Unsatisfiable requirements detected for package FastClosures [9aa1b823]:
FastClosures [9aa1b823] log:
├─possible versions are: [0.1.0, 0.2.0] or uninstalled
├─restricted to versions 0.2.0-* by LinearOperators [5c8ed15e], leaving only versions 0.2.0
│ └─LinearOperators [5c8ed15e] log:
│ ├─possible versions are: 0.5.0 or uninstalled
│ └─LinearOperators [5c8ed15e] is fixed to version 0.5.0+
└─restricted by julia compatibility requirements to versions: uninstalled — no versions left
The latest published version of FastClosures has deprecations and does not pass tests on 0.7 or 1.0, so it seems like it is correctly capped.
There seems to be PR Update to pass on 0.7 by abelsiqueira · Pull Request #8 · c42f/FastClosures.jl · GitHub solving the issues.
Great. Once that is merged, a new version should be tagged which will not be capped. The whole purpose of this process is so that instead of people getting silently broken packages, they get a relatively clear error message indicating that no compatible version of the package exists. So this is an example of things working as they should.
3 Likes
That is a good question and probably I would not have done anything differently. My packages were broken yesterday by API breakages in Interpolations.jl and LinearOperators.jl and the capping thing was then just one more thing.
Your metaphor with the plane is quite good. During the the transition from 0.6 to 1.0 the plane had landed. Probably it would have been better to initiate a new registry for 0.7 / 1.0 where packages needed to register again. But now the plane is flying again, so that seems to be too late for that.
1 Like
That might have replaced the sporadic and short-lived instances of inconvenience users have experienced (like described above) with a lot of work for package authors and maintainers of the registry.
2 Likes
Any package that wanted to work on 1.0 needed work by the package author and the registry maintainers anyway. I don’t see why this would have been more work when a new registry would have been started. And to make it clear: I am currently not talking about the capping thing, which happened right now but my suggestion was, that when a new breaking Julia release is out (i.e. Julia 0.6 → 1.0, Julia 1.0 → 2.0) that packages need to actively acknowledge that they are working with the new version.
2 Likes
As I saw there were some issues opened by you: I would focus on those packages where the most recent version was capped. If older versions are capped and a new version exists and installs just fine, I don’t think one should hassle the package owners about it.
2 Likes
Please uncap the last version of ModiaMath (pull request to METADATA to register the last version). ModiaMath version 0.2.3 supports Julia 0.6, 0.7 and 1.0, but it does not use Compat (see Travis CL for a proof).
I tried to follow your instructions, but I do not have enough julia METADATA/git-knowledge to understand what I have to do. Please, either uncap due to this post or give more detailed instructions how to fill a pull request to uncap ModiaMath.
Note, in the next days also two other packages, Modia and Modia3D with support of Julia 0.6, 0.7 and 1.0 will be registered and none of them uses Compat.
@KZiemian It is good you try to help but remember that opening issues notifies a lot of people, so before opening a lot of them (~180) it is good to verify that this is a good thing to do. Many packages you opened issues on are, for example, not affected by this post.
4 Likes
This may be a bit off topic, but I haven’t found other posts when searching.
I’m quite slow at upgrading packages, since my Julia packages are now maintained only in my (currently quite limited) spare time.
I got around to updating SparseGrids and It Works on my Machine™. After pushing this “upgrade” branch to GitHub my build fails at Travis. As I understand the log, it wants the REQUIRE file. My understanding was that REQUIRE was replaced by the two toml files?
I have probably overlooked something and I’m looking for a TL; DR version of how to upgrade packages to v1.0.
- Use Julia v1.0 package manager instead of “< v0.7”.
- Make the code Work on my Machine.
- ???
Any links or tips are appreciated!
The easiest thing is that you remove the *.toml files again and just use REQUIRE for now, which is currently kept for legacy reasons.
I guess the issue that you are facing is that you have “dev”-ed a dependency within the project which will not work.
Alternatively, would it suffice to simply tag a new version?
Yes
So, for example, if the REQUIRE file lists julia 0.6 (i.e. it claims to support ≥ 0.6) and it was tagged on July 14 or later, then it will be assumed 1.0 compatible, right? (Or only if it uses Compat?)
This PR (when merged) will uncap ModiaMath 0.2.3 since it add an explicit upper bound on the Julia version (< Julia 2.0.0):
The code is here if you want to look at it:
Translating it into words:
- If a package is in the “passing” whitelist, don’t cap its maximal published version.
- If a version has a finite upper bound on Julia, don’t cap it.
- If a version has an interval that only includes 0.7 or 1.1, don’t cap it.
- If a version doesn’t depend on Compat and supports 0.6 or older, cap it.
- If a version claims to support 0.7 or 1.0 and was tagged after July 14th, don’t cap it.
So yes, the last rule means that such a version is assumed to actually be 0.7/1.0-compatible.