It’s actually straightforward to setup GitLab CI runner (not entire GitLab service) on machines you control GitLab Runner | GitLab. It’s easy if you can use docker Run GitLab Runner in a container | GitLab. Once you setup and register the GitLab runner service, you can initiate the job by pushing to gitlab.com. But this requires the runner service to keep running in the background. So I’m not sure if it fits with your need.
FYI, I setup benchmarks on Travis CI and run a couple of benchmarks: Home · TransducersBenchmarksReports.jl. I was worried that it may fluctuate a lot. But it turned out Travis is consistent enough for my need. This is because I only care about the performance relative to the baseline implementation I write.