Testing GPU compatability in CI

AntonReinhard
Wed, 04 Sep 2024 09:04:00 +0000
GPU

I have a package in which I want to add support for some GPU types via package extensions and weak dependencies to them.
Now I want to set up tests that conditionally only run when

1. those GPU packages are installed,

2. those GPU libraries are functional (i.e. they have .functional() = true ), and

3. a GPU of that type is available in the machine/runner (i.e. .devices() is not empty).

Now, 2 and 3 could be realized in code and I could just skip the tests if they are not given.
So I’m really asking about the first point.

My dream state would be being able to have multiple lines in the Project.toml for the tests, such as

[targets]
test = ["SafeTestsets", "Test"]
testcuda = ["SafeTestsets", "Test", "CUDA"]
testamdgpu = ["SafeTestsets", "Test", "AMDGPU"]

And then have directories test, testcuda, and testamdgpu each with their own runtests.jl.
This would have several advantages: A user could just run the tests applicable to their system with the GPU packages/libraries they have installed. Also especially for CI runners it would be nice to only explicitly call the tests that will be able to run (normal tests on a CPU-only runner, CUDA tests on a runner with CUDA and a GPU and so on), both to save time and because I have often seen many errors and/or warnings being thrown while installing the Julia GPU packages when the respective libraries are missing. For example, AMDGPU simply does not support Julia versions before 1.10 at all, so even just trying to install it will fail the CI job, even if I try to check functional() in my tests.

The workaround I’m seeing to achieve something like the testgpu targets from above would be to just manually install the packages in the workflow files, but this seems like manually doing the job that the Project.toml really should be able to do by itself.

Are there any better solutions?

PS: I found these related posts

CI for GPU in Julia, but I’m not asking for runners itself, I have access to those

Testing a package extension?, but there’s no solution

3 posts - 2 participants