Library to track performance over time

I’m often interested in tracking certain metrics’ evolution over time (eg. performance on some benchmark task, model accuracy, time-to-first-plot, testing time, etc.) Is there a library that helps with these things? I know that I can do it myself (write some JSON to a file each time the benchmark is run), but if it’s already nicely packaged, I’d rather use something else.

GitHub - JuliaCI/PkgBenchmark.jl: Easy benchmark tracking for packages may help

1 Like

Neat! But I’m more looking into a year-long timeseries plot showing how, say, train_set_error(...) has decreased over time.

I haven’t continued developing it, but I had started something for that purpose here GitHub - jkrumbiegel/VersionBenchmarks.jl

It was functional but not completely well thought out last I used it.

1 Like