GitLab CI for Julia GPU packages

maleadt · June 25, 2018, 2:45pm

Hi all,

I’ve been testing GitLab CI/CD for GPU-related packages, and despite some missing features it’s been a good experience and I’ve replaced my Buildbot set-up with it. The main advantage is that there is no need for a central configuration, and packages can (much like with Travis) specify a .gitlab-ci.yml with all the relevant instructions to build/test/deploy packages.

As I’ve got some spare processing power, people with a need for GPUs are welcome to use my server for JuliaGPU related testing. It’s a 64-bit Linux-based server with a single GTX 1080p. I’ve documented the procedure to enable GitLab CI/CD + some basic testing template over at GitHub - JuliaGPU/gitlab-ci: Resources related to the JuliaGPU GitLab CI.. For more complicated examples, look at CUDAnative.jl/CUDAdrv.jl/LLVM.jl/CuArrays.jl/ClArrays.jl/Makie.jl

sdanisch · June 25, 2018, 2:54pm

Thanks a lot for doing this @maleadt! This makes testing much easier and improves the stability of Julia’s GPU packages by quite a bit!

juliohm · June 25, 2018, 3:16pm

Is it possible to host a project on GitHub, and test it on GitLab? I have chuncks of code without test coverage because Travis CI doesn’t provide GPUs. Please let me know if you are aware of a solution.

sdanisch · June 25, 2018, 3:17pm

Tim has posted a step by step description:
https://github.com/JuliaGPU/gitlab-ci

juliohm · June 25, 2018, 6:30pm

Thank you @sdanisch, will read it carefully. Good to know it is possible

maleadt · June 26, 2018, 6:24am

The README might not have been clear, but GitLab CI isn’t tied to GitLab (although that integration is obviously the smoothest). As long as your package is reachable by git repository, you can add CI.

maleadt · November 5, 2018, 9:09am

Update for people using this service: due to a GitLab limitation CI testing is restricted to changes on the repo (and not pull requests from forks), but with Bors we can make all changes that need to be tested part of the repo. Documentation has been updated, see CuArrays.jl for real-life usage (eg. this PR).

maleadt · November 22, 2018, 7:45am

Some changes:

updated to CUDA 10
Base images updated to Ubuntu 18.04
new hardware: now contains a GTX 1080 and an RTX 2080 Ti

costrouc · December 21, 2018, 3:49pm

@maleadt we are looking at possibly using gitlab ci for gpu testing for python and c projects. Is there any special configuration that is necessary for to get the gitlab runners working? Is it documented how to setup a gitlab CI runner with gpu support? Currently I see GitHub - JuliaGPU/gitlab-ci: Resources related to the JuliaGPU GitLab CI. provides instructions for the users only

maleadt · December 21, 2018, 7:25pm

The last paragraph lists some details about setting up the runner for GPU, most importantly using runtime = "nvidia". Feel free to contact me if you have any troubles.