I maintain a package (MIPVerify.jl) and largely rely on unit tests in CI + CompatHelper (example PR) to keep my [compat] up to date. Since CompatHelper only adds to the list of versions for each dependency, the lower bounds for many of my package versions are out of date: for example, Memento:0.12 is listed in my Project.toml, but it is in fact incompatible with my minimum listed Julia version of 1.6.
Should I bother updating those lower bounds, or just let the dependency resolver do its magic?
Ideally, you’d have one item in your CI test matrix that specifically tests against the oldest supported Julia version and the oldest version of all your dependencies, as given in the compat-section of your Project.toml. And yes, update all minimum versions when they break (and you can’t/don’t want to fix it)
Well, by “ideally” I meant “I don’t actually do this myself, but I should”
I’m not sure there’s an easy way to do it with the setup you’re using. There should be support for this added to julia-actions/julia-buildpkg. Maybe there already is: it seems the action supports a project variable, so you might be able to point it to a folder where you’ve set up a Project.toml/Manifest.toml with the oldest dependency versions.
Other than that, you’d have to not use the predefined actions for that particular matrix entry and manually set up the appropriate environment and run your tests/runtests.jl.
This might be easier if you define your test environment in test/Project.toml instead of with extras in the main Project.toml (as the manual at some point started recommended, although it seems like the effort to transition from extras to test/Project.toml has fizzled).
At some point, I’ll try playing around with having tests/Manifest-<juliaversion>.toml files which the CI could rename to tests/Manifest.toml to force specific dependency versions in combination with specific Julia versions in the CI
We’ll, by “ideally” I meant “I don’t actually do this myself, but I should”
Ah, I see. Thanks for explaining! Yes, I ideally should too.
Maybe there already is: it seems the action supports a project variable, so you might be able to point it to a folder where you’ve set up a Project.toml/Manifest.toml with the oldest dependency versions.
The file with the oldest dependency version itself risks getting out of sync: imagine modifying the “main” Project.toml but not the test one …
At some point, I’ll try playing around with having tests/Manifest-<juliaversion>.toml files […]
This sounds interesting. Let me know if you make any progress here.
The file with the oldest dependency version itself risks getting out of sync: imagine modifying the “main” Project.toml but not the test one …
Actually, I don’t think so. If you update (increase) the lower bound of one of your dependencies in the main Project.toml but forget to update your CI, you won’t be able to install the package you’re testing into the test environment. So you’ll get a failure, as an instant reminder to update your test.
If you were to lower the lower bound of a dependency, that would be a different matter, but I don’t think that ever happens.
See oldnew_compat.yml · GitHub for a github action that runs tests on oldest/newest version of each dependency (2n runs) or on all versions of each dependency. Uses experimental features of CompatHelperLocal.jl.
