(Continuing the discussion in How to run benchmark CI on every commit? - #3 by tkf)
FYI I just set up CI to run PkgBenchmark.judge https://github.com/tkf/Transducers.jl/pull/32 and I find it pretty useful. Feel free to grab some helper script/setting from there. There were some bits that were not super obvious (like how to invoke git fetch to pull the master branch so that PkgBenchmark can switch to it.).