About checking a package/library `Manifest.toml` into control version (again)

General Usage
, , , ,
13

I might have an idea for getting you the kind of reproducibility you’re looking for without any of the drawbacks of committing Manifest files, and within the limitations of the current Pkg and standard tooling:

  1. Use TestEnv.jl to get a REPL with the test environment of your package. This should work whether you use [extras] or test/Project.toml.
  2. Use Pkg.status() to print the packages and their resolved version numbers in the environment. This would include both explicit and implicit test dependencies
  3. Create a file test/install_supported.jl where you use Pkg.pin commands to force all packages to the exact version shown earlier by Pkg.status(). You could probably write yourself a script to automate this process
  4. At the top of test/run_tests.jl, conditionally include the install_supported.jl file. You could make this conditional on environment variables and/or ARGS: Both Pkg.test and the julia-runtest actions allow passing arguments to Pkg.test
  5. You might consider also creating a file install_lowest_compat.jl where you pin all packages to the minimum version specified in your packages compat bounds.
  6. In CI, create different jobs as desired, e.g. to run the tests without any arguments (with the “latest” compatible packages, what users would generally experience), with the “supported” versions, and the “lowest compat” versions. Similarly, when testing locally, call Pkg.test with the appropriate options the switch between these cases
  7. On a regular basis, e.g., before making a release, update the pinned “supported” versions to the latest versions. This is more tedious that using auto-genererated Manifest files, but presumably something that could be automated.
3 Likes