This is great news, I look forward to trying it out. The tuning time in BenchmarkTools just kills me and I’ve found the system for saving pre-tunes fragile and fiddly.
Have you written about, or could you explain a bit, how Chairmarks achieves consistent benchmarks without all the tuning?
Other question: you say that RegressionTests.jl is still unstable, what are the odds that future breaking releases will be able to read old formats? The thought of having to modify code doesn’t bother me at all but losing access to the old benchmarks, that I would care more about.