Recommendation for CI benchmark to catch regression

jling
Sat, 07 Oct 2023 12:28:44 +0000
Performance

It’s pretty common for Julia package developers to want to monitor key usage latency / performance to avoid regression in PRs. Julia itself has nanosoldier, and some big package such as Makie has their own bot: Hardcode paths for default fonts to avoid search latency by jkrumbiegel · Pull Request #2531 · MakieOrg/Makie.jl · GitHub

Is there a semi-canned solution or workflow people would recommend? I have seen past effort such as: GitHub - maxbennedich/julia-regression-analysis: Regression Analysis for Julia but didn’t seem to be widely used. Maybe it’s just not worth the effort given stability and cost, I think even very popular packages such as DataFrames.jl don’t have automated performance/latency in CI

7 posts - 5 participants