Github Actions workflow for Reverse Dependency Integration Testing

Tooling
,
1

I wanted to share some code that were are using for running reverse dependency tests via GitHubActions.
The idea of reverse dependency testing is to run an integration test using this branch of the project with the downstream code that depends on it.
So we checkout the current master of the downstream project (using a git-checkout action, rather than Pkg), and force it to use this branch of the package being tested via dev.
Then we run the downstream project’s tests; and fail if they fail.

The other trick is we don’t want to fail if this commit is a breaking change and increments the version in the Project.toml to indicate as such.
That should pass, since it won’t break the downstream package, since the package manager won’t let them be installed together.
This is be handled by catching the ResolverError, which even works for test-time dependencies, and exit(0).
Ideally we would give a warning in this case; but since GHA doesn’t really support that, we just pass.
Maybe someone can workout how to express this warning e.g. via posting a comment saying BTW this is not compatible with these packages for these reasons.

The actual code can be found here.
https://github.com/JuliaDiff/ChainRulesCore.jl/blob/b5df374885dbcaacda0f22175064eaa0319cee00/.github/workflows/IntegrationTest.yml
You just want to update the matrix to list the package you want

We are using this in all 4 packages in the ChainRules project; becuase we want to be extra certain not to break peoples stuff.

9 Likes
2

Do you mean that you want a result which isn’t “success” (green) nor “failure” (red), but something intermediate? You can hook into GitHub Actions logger (GitHubActions.jl can do that) to issue a warning, but this won’t be shown anyway in the PR page but in the summary of the job, so I’m not sure it’s what you want

3 Likes