I’m trying to figure out the best practice for reproducible sharing of Julia code for an academic paper. I have a project for which I’m sharing both Project.toml and Manifest.toml to instantiate the exact same package environment for reproducibility. This seems a great setup, but I’ve noticed issues when trying to instantiate a project with a different version of Julia.
For example, if I create a project and add DataFrames in Julia 1.6, when I instantiate the project in Julia 1.5.4 I get:
ERROR: AssertionError: sourcepath !== nothing
In Julia 1.7
┌ Warning: The active manifest file is an older format with no julia version entry. Dependencies may have been resolved with a different julia version.
Which is of course no issue, but highlights that Manifest.toml is changes with Julia versions.
I’ve not checked in the case of DataFrames if adding it in Julia 1.6 installs some package versions incompatible with Julia 1.5.4.
In general, what is the best option here if a set of packages are compatible with two versions Julia but the Manifest.toml files are not, what is the best what to share this? For maximum reproducibility should I restrict the version of Julia?
I think this is telling you that the Manifest needs to be created with the oldest Julia that still makes sense. One gets a warning about the mismatch, but in general things work.
If maximum/exact reproducibility is the goal, using the same version of Julia is implicit, in my opinion. Previous Julia versions are still available for download at Julia Downloads (Old releases), so requiring the same Julia version is not unreasonable.
Also, creating a project in Julia 1.6 and then using in a previous version of Julia (e.g. 1.5.4) should not be expected to work. New (minor) Julia versions are forward compatible, such that existing code/projects (from earlier Julia versions) are expected to continue working, but backwards compatibility (code/projects from newer Julia versions working in older versions) is not a goal of Julia’s Release Process. However, Compat.jl can help with backwards compatibility in some cases.
Also realize that packages can release (non-breaking) new versions that bump the Julia compat dependency, so creating a project in a newer version of Julia, and then trying to use in an older version of Julia could fail due to lacking new features in base Julia, and due to expected package versions not being compatible with the older Julia release.
What I did to enable easy reproducibility was to add a shell script that downloads a specific Julia version in a local folder, setup the correct path, instantiates the current project and runs the program I want to ship.
This allows for a relative small archive to be shipped that can be run by others.
Just make sure to include some checks to avoid downloading Julia every time.
I’ll echo some of the previous comments and suggest that you leave comment noting the Julia version used when constructing the code, such as
# this code was written for Julia 1.6.x
Julia is free to download and easy to install, so I don’t see a problem with suggesting that anybody wanting to run your code as-is must use that specific version. If you had a comment like that and I got errors running your code on Julia 1.5.x, it would be obvious what my next steps should be.
Haha, my next plan as an evil reviewer: “I would like to recommend your manuscript for immediate publication, except I was unable to reproduce your findings in Julia 1.5.4 and JuMP 0.16.”
