Project.toml [compat] best practice: should I manually remove incompatible old dependency versions?

Tooling Package Management
1

I maintain a package (MIPVerify.jl) and largely rely on unit tests in CI + CompatHelper (example PR) to keep my [compat] up to date. Since CompatHelper only adds to the list of versions for each dependency, the lower bounds for many of my package versions are out of date: for example, Memento:0.12 is listed in my Project.toml, but it is in fact incompatible with my minimum listed Julia version of 1.6.

Should I bother updating those lower bounds, or just let the dependency resolver do its magic?

2

If you know that a prior dependency version no longer works, you should probably remove it.

1 Like
3

Ideally, you’d have one item in your CI test matrix that specifically tests against the oldest supported Julia version and the oldest version of all your dependencies, as given in the compat-section of your Project.toml. And yes, update all minimum versions when they break (and you can’t/don’t want to fix it)

1 Like
4

How can I test against the oldest version of all the dependencies? I was searching around for that but couldn’t find the syntax.

3 Likes
5

Very good question! Also curious!

6

Well, by “ideally” I meant “I don’t actually do this myself, but I should” :wink:

I’m not sure there’s an easy way to do it with the setup you’re using. There should be support for this added to julia-actions/julia-buildpkg. Maybe there already is: it seems the action supports a project variable, so you might be able to point it to a folder where you’ve set up a Project.toml/Manifest.toml with the oldest dependency versions.

Other than that, you’d have to not use the predefined actions for that particular matrix entry and manually set up the appropriate environment and run your tests/runtests.jl.

This might be easier if you define your test environment in test/Project.toml instead of with extras in the main Project.toml (as the manual at some point started recommended, although it seems like the effort to transition from extras to test/Project.toml has fizzled).

At some point, I’ll try playing around with having tests/Manifest-<juliaversion>.toml files which the CI could rename to tests/Manifest.toml to force specific dependency versions in combination with specific Julia versions in the CI

7

We’ll, by “ideally” I meant “I don’t actually do this myself, but I should” :wink:

Ah, I see. Thanks for explaining! Yes, I ideally should too.

Maybe there already is: it seems the action supports a project variable, so you might be able to point it to a folder where you’ve set up a Project.toml/Manifest.toml with the oldest dependency versions.

The file with the oldest dependency version itself risks getting out of sync: imagine modifying the “main” Project.toml but not the test one …

At some point, I’ll try playing around with having tests/Manifest-<juliaversion>.toml files […]

This sounds interesting. Let me know if you make any progress here.

To close the circle: I manually removed some packages that were explicitly upper bounded at Julia 0.x (Remove dependency versions compatible with only Julia 0.x by vtjeng · Pull Request #143 · vtjeng/MIPVerify.jl · GitHub), but plan to make no changes otherwise.

8

The file with the oldest dependency version itself risks getting out of sync: imagine modifying the “main” Project.toml but not the test one …

Actually, I don’t think so. If you update (increase) the lower bound of one of your dependencies in the main Project.toml but forget to update your CI, you won’t be able to install the package you’re testing into the test environment. So you’ll get a failure, as an instant reminder to update your test.

If you were to lower the lower bound of a dependency, that would be a different matter, but I don’t think that ever happens.

Let me know if you make any progress here.

Will do!

9

See oldnew_compat.yml · GitHub for a github action that runs tests on oldest/newest version of each dependency (2n runs) or on all versions of each dependency. Uses experimental features of CompatHelperLocal.jl.

10

I finally got around to adding an explicit test for the lowest compat bounds in DocumenterCitations.

The relevant job in the ci.yml is:

  testoldest:
    name: Test Lower Compat Bounds
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: julia-actions/setup-julia@v1
        with:
          version: 1.6
      - uses: julia-actions/cache@v1
      - name: "Instantiate test environment"
        shell: julia --project=test {0}
        run: |
          import Pkg
          println("*** Dev-Installing DocumenterCitations")
          Pkg.develop(path=pwd())
          println("*** Pinning minimal dependencies")
          Pkg.pin([
              Pkg.PackageSpec(name="AbstractTrees", version="0.4.0"),
              Pkg.PackageSpec(name="Bibliography", version="0.2.15"),
              Pkg.PackageSpec(name="Documenter", version="1.0.0"),
              Pkg.PackageSpec(name="MarkdownAST", version="0.1.2"),
              Pkg.PackageSpec(name="OrderedCollections", version="1.6.0"),
          ])
          Pkg.precompile()
          Pkg.status()
      - name: "Run tests"
        shell: julia --color=yes --project=test {0}
        run: |
          include(joinpath(pwd(), "test", "runtests.jl"))

This assumes you have a test/Project.toml file for the test dependencies, although I’m pretty sure you could make it work with TestEnv.jl if you’re using the traditional [extras] in the main Project.toml.

I’m not sure if there’s a way to do this with the julia-runtest Github action that a lot of projects seem to use for their CI testing. I’ve always preferred to explicitly instantiate my test environment and then just run test/runtests.jl, anyway.

11

As it happens I recently made a GitHub action for this: GitHub - cjdoris/julia-downgrade-compat-action: GitHub action to downgrade compat entries before testing

3 Likes