Package version conflicts

General Usage
1

If two packages, say, Flux.jl and Lux.jl, included different versions of Zygote.jl, would this be an issue? Manifest.jl appears to only list one version of each module. What happens in this case? Thanks.

Gordon
2

If they are compatible with a common version of Zygote, version resolution would choose one which works with both. If not you would get an error from the version resolution. There is no way to load two different versions of the same package in the same Julia process.

3

That is what I thought. That makes it increasingly difficult to maintain compatibility as the number of packages increases. It would be nice to have a list of stable packages with changes that happen on a much slower time scale.

4

I really recommend you use environements. For each project/package you are developing, use a distinct environment and most these issues go away.

To do this, you can either start a new environment when starting julia:

pjabardo@i023080L:~/temp/test$ julia --project=.
(test) pkg> st
Status `~/temp/test/Project.toml` (empty project)

Or start Julia normally and activate e new environment:

pjabardo@i023080L:~/temp/test$ julia 
(v1.8) pkg> activate .
  Activating new project at `~/temp/test`

(test) pkg> st
Status `~/temp/test/Project.toml` (empty project)

Now you can add project specific packages. Notice that this will create a Project.toml and Manifest.toml files that contains the specific dependencies used by the project.

Paulo

4 Likes
5

I agree 100% with Paulo. In my run files, I included a few lines to activate the project-specific environment:

# set working directory to directory of current file
cd(@__DIR__)
using Pkg
Pkg.activate("path_to_environment_file")

I also recommend putting compatibility bounds in your Project.toml file. For example

[compat]
some_package = "0.1.0, 0.2.0"

In Julia 1.8 and higher, you can type ] status to see if there are new versions of packages and whether they are outside of your bounds. You can decide whether to increase your compat bounds for new package versions. Switching to this workflow completely eliminated reproducibility problems and compatibility conflicts for me.

In fact, I would say that the Julia’s package system is one of its most underrated features. Good luck doing the same thing with the same effort in R or Python.

1 Like
6

Thanks. To be clear, I have been using environments as you describe, so far without version constraints. In Python, I now use virtual environments and Poetry.

7

In that case, I think your chances of encountering a conflict are low. CompatHelper seems to do a good job keeping packages updated and working together on the development side.

If you add compat bounds based on versions you know to work together, then you can guarantee compatibility and have the control to opt into updates, and revert back if necessary.

Poetry looks quite useful. Thanks for the lead!