StyledStrings and maintainability of reproducible project environments

I have a Julia project environment at LSP-julia/server at main · sublimelsp/LSP-julia · GitHub with a single dependency (LanguageServer.jl) that I update more or less regularly. After updating to Julia 1.11 I noticed that this environment could not be instantiated anymore - because the REPL package, a dependecy of LanguageServer, now needs the new StyledStrings package which is not in Manifest.toml. It worked again after running resolve on Julia 1.11, which added StyledStrings into the Manifest file. However, I saw that StyledStrings has Julia 1.11 as a requirenment StyledStrings.jl/Project.toml at 729f56cb7c7064687a19ed02990a532606d83e4c · JuliaLang/StyledStrings.jl · GitHub, and therefore this would break backwards compatibility of my environment with older Julia versions. As a workaround for now I added a separate Manifest-v1.11.toml file so that it should work on both Julia ≤ 1.10 and on Julia 1.11. Unfortunately I think that this does not scale well; whenever a new Julia version is released I will need to create separate new Manifest-v1.12.toml, Manifest-v1.13.toml etc. files, and even worse, whenever I want to update the project environment I will have to do it separately on each of the corresponding Julia versions.

Is there any better way to solve this situation while keeping backwards compatibility with older Julia versions? I’d like to avoid needing to keep all the previous Julia version installed in order to be able to update this project environment.

For example would it be possible to backport StyledStrings to Julia ≤ 1.10 (if necessary) and loosen its compat requirement to julia = "1", so that I could keep a single Manifest.toml file (in the old manifest format)?

Why do you keep the Manifest.toml files in the first place? Could you just delete them, and not keep them in version control?

Well, releases of LanguageServer.jl are created so infrequently (once or twice per year) that I don’t want to rely on them. Particularly I can’t wait half a year or longer for the next release after a breaking bug was fixed.

Manifests are not compatible between Julia versions. They cannot be, since packages may declare compatibility bounds on Julia itself, leading different versions of Julia to resolve different versions of the dependencies. Only with the support for Manifest-v1.x.toml starting with Julia 1.11 can a single project be resolved for multiple Julia versions simultaneously.

In other words, if this setup has worked in the past I’m afraid you’ve just been lucky, it’s never been supported until Manifest-v1.x.toml came along.

I agree with @nsajko that it seems ill-advised to include the manifest as part of an editor plugin like this. If you want to resolve to the git repo instead of the registry, that’s supported through the [sources] section of Project.toml starting with Julia 1.11: 10. Project.toml and Manifest.toml · Pkg.jl (see also Julia 1.11 Highlights)

In the VS Code extension we ship separate project/manifest combos for each minor Julia version, exactly for this reason (see julia-vscode/scripts/environments/terminalserver at main · julia-vscode/julia-vscode · GitHub for the REPL and julia-vscode/scripts/environments/languageserver at main · julia-vscode/julia-vscode · GitHub for the LS, where we just recently dropped support for running the LS itself on anything older that then current LTS release).

I disagree with that :slight_smile: Users of an editor plugin just want it to work. Anytime you do a new resolve, things might end up in in a state where things don’t work, so I think shipping manifests that pin down versions in a known-to-work state is really, really good for something like an editor plug-in. Yes, it is cumbersome because of the proliferation of versions, but that is painful for the plug-in maintainer, not the end user, which seems a good trade-off :slight_smile: In the VS Code extension we have scripts that keep all of these project/manifests up-to-date.

4 Likes

Sure, if you ship a separate environment for every released minor Julia version like the VSCode plugin, I suppose it’s fine to include the manifest. I was assuming a single project meant to work across multiple Julia versions, as currently done in the OP’s plugin.

2 Likes

Btw. are manifests guaranteed to be portable between platforms? I was imagining that artifact resolution might depend on platform/OS, but maybe not

Yes

Artifacts aren’t part of package resolution.

3 Likes