# PSA: Add Downgrade CI to Better Check Version Compatibility

**URL:** <https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063>\
**Category:** Community\
**Tags:** package, package-manager, semver, versioning\
**Created:** [February 11, 2024, 12:56pm UTC](https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063 "2024-02-11T12:56:01Z")\
**Posts on this page:** 16\
**Page:** 2

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [February 13, 2024, 10:15pm UTC](https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063/21 "2024-02-13T22:15:16Z")

</div>

> [@cjdoris](#):
>
> CI is too expensive to be testing lots of combinations of old versions of dependencies.

Well, running tests n (= number of direct deps) times once in a while is definitely feasible. The benefit is that such a run directly indicates which compats to tighten.  
Not for every commit, here I agree!

For example, I ran this job on Copulasjl (registered version), and it found definite incompatibility with TaylorSeries = 0.12.0. Some other downgrades also failed tests, but seemingly because tests for that package are non-determenistic and can just randomly fail.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [February 13, 2024, 10:19pm UTC](https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063/22 "2024-02-13T22:19:33Z")

</div>

> [@cjdoris](#):
>
> And you’ll need to do that anyway in order to work out what the lower bound should be unless you’re going to bisect your way there or just set it to the latest version.

That same CI job ([oldnew\_compat.yml · GitHub](https://gist.github.com/aplavin/ec1024d00db6ebe8a7fa0a38aa8cca6c)) can also test _all_ versions of each dependency (: Just specify “all” instead of “two” when running the workflow.  
This is supposed to be ran rarely: after you identified that _some_ compat versions don’t work, and need to establish what exactly are those. Then, the result directly shows how compat bounds need to be tightened.

---

<div class="post-metadata">

**Author:** ![e3c6](https://avatars.discourse-cdn.com/v4/letter/e/e79b87/32.png) [@e3c6](https://discourse.julialang.org/u/e3c6)\
**Post date:** [February 13, 2024, 10:28pm UTC](https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063/23 "2024-02-13T22:28:46Z")

</div>

OP should probably update to the new [GitHub - julia-actions/julia-downgrade-compat: GitHub action to downgrade julia compat entries before testing](https://github.com/julia-actions/julia-downgrade-compat).

---

<div class="post-metadata">

**Author:** ![e3c6](https://avatars.discourse-cdn.com/v4/letter/e/e79b87/32.png) [@e3c6](https://discourse.julialang.org/u/e3c6)\
**Post date:** [February 13, 2024, 10:31pm UTC](https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063/24 "2024-02-13T22:31:33Z")

</div>

Could this be in a package? Like Aqua? So that I can also run this locally, like `Aqua.test_downgrade(MyPackage)`

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [February 14, 2024, 3:39am UTC](https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063/25 "2024-02-14T03:39:50Z")

</div>

> <https://github.com/JuliaHEP/UnROOT.jl/actions/runs/7896149188/job/21549638976?pr=307#step:4:32>

there seems to be some parse error?

---

<div class="post-metadata">

**Author:** ![mestinso](https://avatars.discourse-cdn.com/v4/letter/m/bc79bd/32.png) [@mestinso](https://discourse.julialang.org/u/mestinso)\
**Post date:** [February 14, 2024, 4:28am UTC](https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063/26 "2024-02-14T04:28:46Z")

</div>

Yes, I too would be interested in whether there is a way to do this check locally with no CI in the loop…

---

<div class="post-metadata">

**Author:** ![lrnv](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lrnv/32/19373_2.png) [@lrnv](https://discourse.julialang.org/u/lrnv)\
**Post date:** [February 14, 2024, 8:36am UTC](https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063/27 "2024-02-14T08:36:43Z")

</div>

> [@aplavin](#):
>
> For example, I ran this job on [Copulasjl](https://juliahub.com/ui/Packages/Copula) (registered version), and it found definite incompatibility with TaylorSeries = 0.12.0. Some other downgrades also failed tests, but seemingly because tests for that package are non-determenistic and can just randomly fail.

Can i see these results somewhere ?

---

<div class="post-metadata">

**Author:** ![ianshmean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ianshmean/32/216042_2.png) [@ianshmean](https://discourse.julialang.org/u/ianshmean)\
**Post date:** [February 14, 2024, 10:19am UTC](https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063/28 "2024-02-14T10:19:11Z")

</div>

IIUC this is an issue with parsing underscores in the name. It’s fixed on v1.0.7 but the v1 tag is stuck at v1.0.6

On the new julia-actions/julia-downgrade-compat fork (note the rename) v1 points to v1.0.7. You might just want to try the fork out?

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [February 14, 2024, 4:09pm UTC](https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063/29 "2024-02-14T16:09:36Z")

</div>

> [@lrnv](#):
>
> Can i see these results somewhere ?

I have a separate private repo with this CI job, that I can run for any registered Julia package – don’t know how to share its results specifically. Probably the easiest is to just run it, no setup is needed whatsoever.  
Here’s a screenshot:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/7/e/7e7745fa42f52a016f624cbb27abefe2b2c074d8.jpeg)  
It indicates the methoderror happening when TaylorSeries are downgraded.  
For other deps downgrades, it seems like failures are just random due to non-deterministic tests. See eg  
 ![image](https://global.discourse-cdn.com/julialang/original/3X/4/1/413c0d093514a381a7af53a974b6bc4d4c1a431b.jpeg)

I’m also not too familiar with sharing github actions properly, so simply uploaded and linked it as a yml file above.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [February 14, 2024, 7:06pm UTC](https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063/30 "2024-02-14T19:06:48Z")

</div>

![image](https://global.discourse-cdn.com/julialang/original/3X/9/f/9f8c6621bf94abf42a1b2b6c0a5694f1c4fa0ad9.png)

does it not work with stdlib versions? I guess they need to be included in the `skip` list

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [February 15, 2024, 10:28am UTC](https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063/31 "2024-02-15T10:28:37Z")

</div>

@cjdoris I think it would be good to have it find the lowest resolvable minor though, since a lot of packages just choose a major `PkgA = "6"` and that effectively lower bounds it to the lowest minor that is possible through other dependencies. Testing it on forcing `6.0.0` is a bit stricter than what Pkg can actually do. It is nice to get this all handled but I think the less strict version would still improve the ecosystem but be much easier to get passing.

---

<div class="post-metadata">

**Author:** ![cjdoris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cjdoris/32/213133_2.png) [@cjdoris](https://discourse.julialang.org/u/cjdoris)\
**Post date:** [February 15, 2024, 7:05pm UTC](https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063/32 "2024-02-15T19:05:48Z")

</div>

The trouble with that is that there can be multiple sets of minimal resolvable packages (exponentially many in the worst case) so what do we do then? Just pick one?

The current approach has the advantage of being declarative - you can tell what the minimum versions that will be tested are just by reading Project.toml.

Also the current approach is a very simple modification to Project.toml (you could probably write it in sed).

I appreciate the point about making it easier on users to get this test passing the first time they use it. Probably for those, using aplavin’s recipe as a one-off will get them there - I should add this to the docs for the action.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [February 18, 2024, 10:16am UTC](https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063/33 "2024-02-18T10:16:51Z")

</div>

I tried out the downgrade CI on some simple packages with few deps and it was great, but trying to use it on a big package with many deps just felt like a total waste of time. It needs to do something smarter than just pinning everything to the minimum versions.

> [@cjdoris](#):
>
> The trouble with that is that there can be multiple sets of minimal resolvable packages (exponentially many in the worst case) so what do we do then? Just pick one?

This is the exact same problem as trying to install the newest available versions of packages, just flipping the sign of the inequality operator.

Presumably we could just re-use a lot of the resolver code from Pkg, but just flip the comparison operator so it tries to find the oldest installable set of packages instead of the newest installable set.

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [February 18, 2024, 6:25pm UTC](https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063/34 "2024-02-18T18:25:16Z")

</div>

The same situation can happen with the normal operation as well, right? The newest version of each package could be incompatible just like the lowest compatible versions. How does the tiebreaker work in that case?  
No matter how it works, I agree that we should be able to reuse that logic and just swap the comparison operator as the situation should be symmetric.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [March 14, 2024, 1:22pm UTC](https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063/35 "2024-03-14T13:22:42Z")

</div>

Since Julia 1.9 (or was it 1.10?), we need to declare the versions of the Base dependencies, like “Statistics”, “Printf”, etc, to the same version as the Julia one declared. So for instance we can have something like this in the Project.toml:

```julia
[compat]
Printf = "1.9"
Julia = "1.9"

```

This causes some trouble in this downgrade-package test, because I can only test the packages in the minimal Julia version possible, which most likely won’t be the most used one. Am I missing something? I think it would be useful to be able to test with the minimal versions of packages _in the current stable Julia version_.

Currently the test fails right away if I want to test the package in Julia 1.10, since it won’t be able to install Printf 1.9.

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [March 14, 2024, 1:45pm UTC](https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063/36 "2024-03-14T13:45:53Z")

</div>

I think there’s an option to skip stdlibs:

```julia
- uses: cjdoris/julia-downgrade-compat-action@v1
  with:
      skip: Printf

```

[Previous page](https://discourse.julialang.org/t/psa-add-downgrade-ci-to-better-check-version-compatibility/110063.md?page=1)