I’m often interested in tracking certain metrics’ evolution over time (eg. performance on some benchmark task, model accuracy, time-to-first-plot, testing time, etc.) Is there a library that helps with these things? I know that I can do it myself (write some JSON to a file each time the benchmark is run), but if it’s already nicely packaged, I’d rather use something else.
Neat! But I’m more looking into a year-long timeseries plot showing how, say, train_set_error(...) has decreased over time.
I haven’t continued developing it, but I had started something for that purpose here GitHub - jkrumbiegel/VersionBenchmarks.jl
It was functional but not completely well thought out last I used it.
1 Like