I’m using Pkg.dev to develop a package (an application, not a library) on a local path. I commit the package Manifest into my repo so I can revert and run instantiate in case I mess up my package dependencies.
That happened today. After an update some images my package creates using GeoMakie became a bit messed up (I got a tiny plot in the upper left corner on a large white image). I didn’t have time to figure out what had changed so I decided to just revert for now.
So I checked out a commit with a Project.toml and Manifest.toml from a previous working state, then activate MyPackage and instantiate but the problem remained. Even earlier commits gave the same result. Confused I tried lots of weird stuff, but the only thing that fixed the problem was copying over my old 1.4 default environment into the 1.5 folder (followed by activate and instantiate).
This raises three questions:
-
I don’t understand how copying the 1.4 environment could work. In my limited understanding of Pkg, every package has its own environment that graphs all its dependencies. The default environment is only used for the REPL and scripts that run outside of packages. Environments don’t interact. Am I wrong here?
-
The only thing I can think of is that the package Manifest I’ve committed was out of sync. There’s a paragraph towards the end of this section of the Pkg docs that suggests this happens every time you add dependencies to a package linked with dev. Is this correct? Does that mean that I should always run resolve before I commit to keep my package Manifest in sync?
-
Finally: if the Manifest sometimes becomes out of sync with the actual dependency graph and therefore can’t always be trusted, how can you check what dependency versions are really active?
I think something else must have happened when the problem got fixed, eg you restarted Julia at some point so the relevant package versions could be loaded.
In any case, instantiate is the recommended way to do what you want. It is always possible (but unlikely) that it had a bug, but to investigate we would need a way to reproduce what you did, for which we lack sufficient information. Eg when you say that “the problem remained”, it is not clear to me what the problem actually was.
Thanks. When I said “the problem remained” in that quote, I meant that the updated dependency that triggered the graphical bug appeared to have remained installed in my package, despite that I rolled back the repo to a pre-update commit, ran activate MyPackage and instantiate - and yes, restarted Julia several times.
The updated dependency remained “magically” active in my package until I (out of desperation) replaced my 1.5 default environment with the old 1.4 version. But that’s the mystery to me - I thought the Project.toml and Manifest.toml contained references to everything needed in the package dependency hierarchy. Once I instantiated that, how can modifying the default environment change what versions my package used?
I am not sure why you think that this is what happened. It does not necessarily follow from your description.
In any case, if you think there is an issue please try to make an MWE (eg an otherwise empty project with a project and manifest file) that can reproduce this.
Granted, I was tired and a bit stressed yesterday over having to deal with update issues when I didn’t have time to do so, so I suppose it’s possible that I’m mistaken about some link in the event chain described here. I’ll try to reproduce it in an MWE.
But can I interpret your responses to mean that you are just as surprised about what I describe above as I am? Because I was beginning to suspect that my mental model of Pkg was completely wrong, so it would be a consolation if I didn’t have to start relearning everything from scratch.
I just don’t have enough specific information to be surprised — I am still not sure what happened.
Major bugs in Pkg happen, but not very often. Personally, if I get things working and I have reason to assume that I was really tired and made a mistake somehow, I usually do not pursue these things. YMMV.
The procect/manifest of the current active environement used. Everything else is ignored (with the exception of loading packages in the dead ült environment in the REPL).
So if had the default environment active, only the manifest there is relevant.
If yiu want to see the version of all packages that will get loaded you can do pkg> st - m
3 Likes
Great, that was helpful for my understanding and the st -m is something I think I’ll use a lot in the future.
I finally solved the issues I’ve been having - everything began working again as expected after I forced a garbage collection with Pkg.gc(;collect_delay=Hour(0)). Still not sure what happened, but if there’s a problem I can reproduce I’ll open an issue. Thanks for helping Tamas & Kristoffer!
That’s surprising since GC shouldn’t affect any behavior. Oh well…
1 Like
Well, the full chain was this. Pkg wouldn’t let me install a particular version of a package, because of “Unsatisfiable requirements” - it said “restricted to version a.b.c by an explicit requirement”. It’s true that I had previously put =a.b.c into [compat], but I had already removed that constraint and ran update and resolve. But somehow the requirement remained until I removed it entirely and then ran gc as above. Then finally I could re-add it and update to a version that was compatible with everything else.
I’m going to test to see if the same zombie requirement happens with an MWE.