How to achieve consistency when using workspaces on Julia 1.11?

In my project, I have a couple of sub-projects:

[workspace]
projects = ["examples", "examples_3d", "docs", "test"]

I am trying to write an install script that instantiates and precompiles the main project and all sub-projects. It seems to work with Julia 1.12, but not with Julia 1.11. Is there a way to make it work with Julia 1.11?

I tried:

  1. First to copy Manifest-v1.11.toml.default to Manifest-v1.11.toml (the de9.fault manifest is from Git and represents the last known good set of package versions
  2. Instantiate the project
julia --project -e '
using Pkg
try
    Pkg.instantiate()
catch e
    @warn "Pkg.instantiate() failed, attempting fresh resolve..." exception=e
    proj_dir = dirname(Base.active_project())
    # Remove all manifest files to force a clean resolve
    for f in readdir(proj_dir)
        if startswith(f, "Manifest") && endswith(f, ".toml") && !endswith(f, ".default")
            rm(joinpath(proj_dir, f), force=true)
        end
    end
    Pkg.resolve()
    Pkg.instantiate()
end
'
  1. Precompile the main project:
julia --project -e 'using Pkg; Pkg.precompile()'
  1. Resolve sub-projects independently:
    rm -f examples/Manifest*.toml test/Manifest*.toml examples_3d/Manifest*.toml
    julia --project=examples -e 'using Pkg; Pkg.resolve(); Pkg.instantiate()'
    echo "Precompiling examples project..."
    julia --project=examples -e 'using Pkg; Pkg.precompile()'
    julia --project=test -e 'using Pkg; Pkg.resolve(); Pkg.instantiate()'
    echo "Precompiling test project..."
    julia --project=test -e 'using Pkg; Pkg.precompile()'
    julia --project=examples_3d -e 'using Pkg; Pkg.resolve(); Pkg.instantiate()'
    echo "Precompiling examples_3d project..."
    julia --project=examples_3d -e 'using Pkg; Pkg.precompile()'

While this somewhat works, it results in:

  • the sub-projects not using the same package versions as the main project
  • If I open any of the examples, it starts precompiling the packages again

Any idea how to fix these issues?

Observation:

  • If I copy the main manifest into a subproject, activate the subproject, and run Pkg.resolve(), this often fails. Is this a bug in the Pkg package?

UPDATE:
I created an issue: Resolve does not find a solution · Issue #4623 · JuliaLang/Pkg.jl · GitHub
Which was closed, so I created a new issue:
Resolve not working as intended · Issue #4624 · JuliaLang/Pkg.jl · GitHub

If I install the project mentioned above with:

git clone https://github.com/OpenSourceAWE/KiteModels.jl.git
cd KiteModels.jl/bin
./install
cd ..

and then run julia with

./bin/run_julia

and then execute

using Pkg; Pkg.activate("examples"); using ControlPlots, DSP

Then 6 packages get pre-compiled:

julia> using Pkg; Pkg.activate("examples"); using ControlPlots, DSP
  Activating project at `~/repos/KiteModels.jl/examples`
Precompiling ControlPlots...
  6 dependencies successfully precompiled in 27 seconds. 46 already precompiled.
Precompiling JSONArrowExt...
  1 dependency successfully precompiled in 5 seconds. 12 already precompiled.
Precompiling SciMLBasePyCallExt...
  1 dependency successfully precompiled in 4 seconds. 71 already precompiled.
Precompiling UnPackExt...
  1 dependency successfully precompiled in 1 seconds. 35 already precompiled.
Precompiling DSP...
  3 dependencies successfully precompiled in 18 seconds. 58 already precompiled.
Precompiling PolynomialsChainRulesCoreExt...
  1 dependency successfully precompiled in 1 seconds. 24 already precompiled.

If I restart Julia and execute the same command again, they get precompiled again.

Why?

What can be the reason the precompiled binaries do not survive a restart of Julia?

I can’t imagine this working without workspaces, which were introduced in 1.12. I haven’t heard of any efforts to backport it, which is rare for features generally, but it’s very odd the Pkg docs don’t mention anything to the effect of “1.12 onwards”.

This is a fundamental problem which can’t be solved by copying manifestes. Consider the following:

  • root dir env only depends on PkgA with compat=[1,2]. If you resolve this env, you will get PkgA at version 2.
  • sub env depends on PkgA and PkgB. PkgB is only compatible with PkgA@v1. Therefore, you sub env will get PkgA@v1.

Both of those are resolved correct, but they are just incompatible.
The “workspaces” feature (which certainly won’t be backported) essentially creates a virtual environment which merges all dependencies and compats of all sub envs. Therefore, you will get PkgA@v1 even in the root dir.

If you want similar behavior, you could create an additional dev env for 1.11:

root
  - Project.toml
  - subenv/
    - Project.toml
  - dev_1.11_env/
    - Project.toml # <- autogenerate

where you essential autogenerate a Project.toml by parsing the sub envs and combining all dependencies. Then you’d need to make sure that you always start your julia 1.11 sessions like julia --project=/path/to/dev_1.11_env. This dev env would be a true superset of all subenv envs and therefore could run all code in the subenvs while caching the precompilation.

Caveats:

  • the merging of the differen Project files might be non-trivial, especially if compat bounds are involved
  • you won’t be able to “add” packages to the subenvs, because you never activate them

The other obvious alternative is to resolve all envs separatly and precompile all envs separatly once during ./install. This takes longer, but no matter what subenv julia --project=../subenv activates, there will be a valid cache for that env already.

Thanks for your input!

I think I have it working now. What I do is:

  • delete and re-add the main registry (yes, it is sometimes needed, students have computers where it is broken)
  • copy Manifest-v1.11.toml.default to Manifest-v1.11.toml
  • resolve and instantiate the main environment in a try-catch block; if it fails, delete the manifest and resolve and instantiate again. In that case, rename Manifest.toml to Manifest-v1.11.toml
  • copy Manifest-v1.11 to all sub-environments, but not to docs environment
  • do the same in each sub environment
    So far, this seems to work on Linux and Windows, tests on MacOS still ongoing.

I can now run examples from each sub-environment and nothing gets pre-compiled, all the compilation happens in the install script.

I understand that this might not work well if there are conflicting requirements for the sub-projects. But in my project, this should not be the case.

I had to manually downgrade SciMLBase from 2.144.2 to 2.144.0 in Manifest-v1.11.toml.default, though.