Is it possible - and if yes, how - to stack multiple environments for a reproducibility repository? That is, I would like to have two sets of
Manifest.toml files living outside the
DEPOT_PATH where I am able to independently (and in a reproducible manner!) control the package versions used.
Here is my use case (hopefully this makes it clearer):
To a reproducibility repository, I have added an environment A with a
Manifest.toml pair that specifies two packages that are used for my workload, OrdinaryDiffEq.jl and Trixi.jl, including their dependencies. I would like to add a second environment B that only contains PackageCompiler.jl that allows one to compile a certain workflow using the packages from environment A.
Now, I could add PackageCompiler.jl directly to environment A, but I do not want to do this, since it might pull in additional dependencies that most people do not care about. Thus I would like to mimic the “standard” Julia setup where I have an active project (in this case, A) but where I am also able to load also packages from the default environment (which is in the
DEPOT_PATH). The difference here is that I would like to put the “default environment” also in the reproducibility repository and have the users instantiate and use this independently.
You can append the PackageCompiler project to the installation instructions/
make target for creating the sysimage. See for example https://github.com/fredrikekre/.dotfiles/blob/master/.julia/environments/nvim-lspconfig/Makefile#L52-#L53.
Ah, do you mean one should be modifying the
LOAD_PATH by setting
JULIA_LOAD_PATH appropriately? E.g., with a directory structure as
I would first instantiate both projects individually by running
julia --project=project_A -e 'using Pkg; Pkg.instantiate()'
julia --project=project_B -e 'using Pkg; Pkg.instantiate()'
and then I could stack the environments by setting
JULIA_LOAD_PATH, e.g., like
JULIA_LOAD_PATH="$(pwd)/project_A:$(pwd)/project_B" julia -e '...'
project_A would now be the primary environment, and
project_B the secondary. At least that’s what I pieced together from looking at Code Loading · The Julia Language and Constants · The Julia Language.
EDIT1: Simplified the
@stdlib as suggested by @fredrikekre
EDIT3: Accept this answer incorporating the help from @fredrikekre to make it easier for future solution seekers.
Also, don’t include
@stdlib there, add necessary stdlibs to the project environments instead.
Thanks a lot for your support!
I included your suggestion in my example above to have everything in one piece. Thanks!
For the specific case of PackageCompiler you don’t need to stack them though. You can use the
project keyword argument instead. In this case you can start Julia in the PackageCompiler project, and then pass the real project to