Background: I’m developing some code on my macbook in the Julia release channel (1.11.2) and also using my university’s HPC which runs Julia 1.10.2.
I’m running all the time into this kind of problem during development. Let’s say I commit some new code on the repo from my macbook. Then, pulling changes from the HPC and trying to run the code gives me this:
julia> using FlowComplexity
┌ Warning: The active manifest file has dependencies that were resolved with a different julia version (1.11.2). Unexpected behavior may occur.
└ @ ~/repos/flow-complexity/Manifest.toml:0
[ Info: Precompiling FlowComplexity [d568c949-88fe-501e-a4ba-7edaec872f9a]
ERROR: LoadError: ArgumentError: Package Statistics [10745b16-79ce-11e8-11f9-7d13ad32a3b2] is required but does not seem to be installed:
- Run `Pkg.instantiate()` to install all recorded dependencies.
Stacktrace:
[1] _require(pkg::Base.PkgId, env::String)
@ Base ./loading.jl:1926
...
The solution that I’ve found works more or less is to do Pkg.update() on the machine where the code doesn’t compile (e.g. the HPC), then the code usually works and the Manifest.toml file gets updated. I commit this back to the repo, pull the changes on my macbook but then get a similar error! I do Pkg.update() again, this time on my macbook, commit the manifest, run the code on the HPC. Sometimes it works, sometimes I need to do a few more Pkg.update() back-and-forth like this.
This is so time consuming I can’t believe:
this is the way it’s supposed to work
only a minor difference in Julia version (1.11.2 vs 1.10.2) would imply so much incompatibility
And then what if I want to run the code on another machine with yet another version of Julia which I can’t update/modify? There has to be a way to run a given project on slightly different Julia versions!
Save the version that works in v1.10 as the usual Manifest.toml, and save the version that works in v1.11 as Manifest-v1.11.toml. Then Julia v1.11 will never touch the former file and v1.10 will never touch the latter file, for any Pkg operation.
Not exactly sure what you are doing or need, but just pointing out that packages shared with the community lack or ignore a Manifest.toml because it must vary so much more than Project.toml. Manifest files change with patches of installed packages, changing the Julia version is more drastic than that. Committing manifest files does have its uses for tests and reproducibility though.
The simplest solution is probably not to check in Manifest.toml in the repo. Assuming it’s a git repo, run git rm Manifest.toml and add “Manifest.toml” to .gitignore.
I would just use exactly the same Julia version on the cluster and on your own computer when you work on that project. With Juliaup you can have multiple Julia versions installed at the same time, so on your own system you can just use Julia 1.10 for this one project, you don’t have to switch everything back to the old version.
Julia has great Project + Manifest reproducibility, but it only applies when using the exact same Julia version – either 1.11 or 1.10 in your case.
On different versions, there’s no guarantee one can instantiate the manifest, exactly as you see in the error message.
Project.toml should often be possible to instantiate even on a slightly different version. It may easily result in different package versions though, so make sure to put your constraints into the [compat] section.
@davidanthoff yeah I’ve considered this, but let’s say I want to run simulations on yet another machine/cluster/so on, and that this other environment has yet another version of Julia which differs from that of cluster #1. What am I gonna do?
@aplavin and to everyone else: thanks for explaining this. Basically, I thought that I’d indeed sacrifice exact reproducibility by not committing the manifest file as @danielwe suggested. But if keeping only the project file is the only way to ensure some level of reproducibility between Julia versions, then so be it.
greatpet pointed out the relatively new feature of Julia-versioned manifest files, so if you want to lock down the dependencies’ versions per Julia version, you can. A more flexible project file is just preferred if you distribute your package to the world and want to let people benefit from compatible patches and updates to the dependencies. That’s not always the case, even if you’re sharing things e.g. one-and-done notebooks that need to be perfectly reproduced forever.
“Exact reproducibility” by definition means that you’re going to run the same code in exactly the same environment. That definitely includes “same Julia version”. That is, if I wanted to reproduce the results of a paper where the code was versioned in git, I would still make sure to install the exact same versions of all the software / compilers they used and run it on the same platform. Once you start changing these, you move towards “replication”, where (if the original result is valid) you should get the same answers within statistical margins with different compilers, or different numerical parameters like grid spacing, or increasingly larger deviations for the original work, up to and including fundamental methodology.
In a research project, it makes sense to commit the Manifest at the very end to allow for exact reproducibility. But generally, the Project.toml should have enough granularity to ensure reproducibility/replicability with slight variations, such as the Julia version or the OS platform. If you want (and that might be a good idea), you can take extra care to pin exact versions of your dependencies in the Project.toml. If you want to go even further, you can add and pin your indirect dependencies in the Project.toml as well. That sometimes makes sense for particularly “important” indirect dependencies. E.g., something like Zygote, since Zygote is notorious for correctness bugs, and you’d want to protect yourself against regressions.
The more you pin, the more you lose flexibility. If you’re developing a package_ that you want others to use, as opposed to on-off “research project code”, then you generally want to keep dependencies as wide as possible, and rely on testing to ensure compatibility. I wouldn’t commit manifests in that scenario.
To emphasize the crucial point made by @goerz: by running on different Julia versions, you’re already sacrificing exact reproducibility. If you want exact reproducibility you should indeed commit the manifest, but then you must also ensure that you’re always running on the same Julia version.
If you install Julia via the official recommended way these days (which is Juliaup), then installing yet another version of Julia on this other cluster is just juliaup add 1.10 or something like that. In general, the hope here is that installing another Julia version is no more involved than installing another package.
We are also working on a feature for Juliaup where this kind of model is even more strong: we want to get to a situation where if you just run julia with a project, it automatically launches (or installs, if necessary) the exact Julia version that is recorded in the manifest, just like at the moment a manifest already pins down exact package versions.
If you install Julia via the official recommended way these days (which is Juliaup), then installing yet another version of Julia on this other cluster is just juliaup add 1.10 or something like that. In general, the hope here is that installing another Julia version is no more involved than installing another package.
Unfortunately juliaup isn’t available on the HPC I’m working on. Not sure how much additional work would be required to install it/other Julia versions as I don’t have elevated privileges on this environment.
We are also working on a feature for Juliaup where this kind of model is even more strong: we want to get to a situation where if you just run julia with a project, it automatically launches (or installs, if necessary) the exact Julia version that is recorded in the manifest, just like at the moment a manifest already pins down exact package versions.
Manifests are portable across operating systems but not across Julia versions. (Patch versions should usually be fine, but minor releases generally won’t be.) This is fairly unavoidable because different versions of packages are compatible with different versions of Julia. How could we possibly guarantee that a single manifest would work with all possible versions of Julia?
Summary of solutions in this thread:
Don’t check your manifest into git. This does sacrifice exact reproducibility, but if you’re using different Julia versions you don’t have exact reproducibility across those anyway.
Use versioned Manifest files. Since Julia 1.11 (I believe), if you have a file named Manifest-vX.Y.toml then it will preferentially be used by Julia vX.Y over Manifest.toml, so you can have different manifests checked into your repo for different Julia versions. This lets you check in your manifests and use multiple Julia versions without having to re-resolve all the time.
Use juliaup to install and manage Julia versions for you and use the same exact version on each machine. This does not require admin permissions so anyone can do it in their own user account.
Building on the last option, you can even use the JULIAUP_CHANNEL environment variable to control what version juliaup starts by default; along with a tool like direnv this lets you automatically start the right version when you’re in the project directory, so you can smoothly use different Julia versions in different projects.
This last approach gives the most reproducibility and arguably the most convenience since it controls and versions both the Julia version and the exact package versions, you only need one manifest, and you don’t have to constantly reresolve.
But even this can’t be guaranteed, right? A package release may set a lower bound on the Julia patch version if it depends on a bugfix from that patch. So a manifest created by Julia 1.x.y should work for Julia 1.x.z where z >= y, but not necessarily where z < y.
juliaup override makes it easy to do that without any third-party tools. But really, we should merge this PR that uses the manifest to select the Julia version, IMO that will be by far the best/cleanest solution to all of these problems.
