Should we have a package update PSA?

Community
1

I’ve been tinkering with the idea of writing this for a while but never got around to it, so I’ll at least put the idea here in case anyone agrees and wants to make a start. I believe over the last few months the most frequently repeated question is “I can’t update package X to its latest version”, and always invariably the answer is:

  1. Do ]add X@versionyouwant to find out what’s holding you back; and
  2. Work with environments, as you actually don’t need ObscurePackageHoldingEverythingBack which you added to your default environment 3 years ago most of the time

Now of course there are nuances to this (firewalls, package servers down/out of sync with registry, actual dependency conflicts in an environment where both packages are required for the task at hand) which should ideally be covered, but in the first instance I feel like it would be good to have this as a PSA to link to (and hopefully for people searching for “unsatisfiable requirements…” to stumble on before posting)

I believe it’s possible to make Wiki posts here which multiple users can contribute to, which might be a good option for this? I have no idea how that works though.

11 Likes
2

I think that the ideal place for this is FAQ/tutorial sections in the Pkg manual, eg on conflict resolution (1 above) see

https://pkgdocs.julialang.org/dev/managing-packages/#conflicts

Contributing more sections on common problems and how to deal with them would be extremely helpful.

Using environments for anything cannot be over-emphasized. They are comparable to branches in git, cheap and meant to be created all the time without hesitation.

5 Likes
3

I would even argue that julia envs are lighter and can be more common than branches in git (: Personally I almost never use branches in one-man projects, but still create environments for them.

4 Likes
4

I haven’t had the same smooth experience using environments, so maybe someone can help.

I have package conflicts all the time and it’s extremely disruptive to my workflow since it often takes many minutes to find a non-conflicting versionset that does what I need.

To fork my current git environment I do git checkout -b newbranch. What is the equivalent for julia environments? ]activate newenv isn’t equivalent because

  • it doesn’t include the existing packages, so I need to figure out exactly which packages I need again (plus I forget one, get an error, and have to install it, repeat)
  • it doesn’t fork the runtime, so I have to wait for code loading on my script in every new environment
  • Revise doesn’t handle updated or removed dependencies

Git also remembers which branch is active across invocations, while Julia doesn’t, so I have to remember to activate the right environment each time.

Am I missing something here, or can I make my workflow more convenient?

2 Likes
5

You can copy the Project.toml and Manifest.toml files from the “base” environment to the “forked” one. If it’s in a different computer, you should call ]instantiate the first time you activiate the new environment, to ensure that all direct and indirect dependencies are installed and built.

I’m not aware of any built-in solution to do that. A dirty workaround could be using a custom function to activate projects, which in addition to calling Pkg.activate, writes the path of the target in the JULIA_PROJECT environment variable of your system – such that it persists across sessions. That custom function might be written in your startup.jl file, such that it is always available.

Also, there are tools which give you that out of the box, e.g. if you work with the VS Code extension, by default it starts on the last workspace, and if there is a Project.toml file in it, that environment is activated at startup.

3 Likes
6

Just to nitpick for those that missed your smiley - Julia’s environments are really light but there’s nothing lighter than a git branch. They’re literally just a pointer (to a commit). Watch Git For Ages 4 And Up, it’s one of the best computing talks I’ve ever seen.

1 Like
7

Every bit of doc will help here. Not a day goes by without a package being downgraded.

I don’t have a great workflow. I have a workspace environment I clean up regularly and in which I always install new packages I’m trying out. In my .julia/confi/startup.jl I have the following:

cd("/path/to/workspace")
import Pkg
Pkg.activate(".")

to make sure that I don’t accidentally install a package in the wrong environment.

For my own packages I have a script with Pkg.add["this", "that"] which I run once in a while after deleting most of the stuff in the Project.toml and Manifest.toml, in order to start from a clean slate. For instance when Julia 1.6.1 was released a few days ago, I did a cleanup.

When a package I use for a small feature induces downgrades (some small packages are rarely updated), I download the file containing the methods I need and keep that inside my package. I feel like a thief everytime, but it saves a few headaches.

8

It doesn’t fulfill all your requirements but I have my Julia environment checked in to my git repository and start Julia with julia --project (assumes you have your current directory in your project). Then forking my Julia environment is literally git checkout -b newbranch and restarting Julia.

For throwaway experiments with new packages I can heartily recommend
pkg> activate --temp

5 Likes
9

Maybe something like ] activate --fork which does ] activate --temp and copies over the previously active environment (toml files) would be worth having? Should be easy to add I presume.

2 Likes