Is anyone aware of a Julia package that implements testing of time series output against baselines?
By this I mean that I’d like to be able to compare simulation outputs over time against some “truth model” equivalent outputs, where the comparison isn’t done simply pointwise but can also tolerate some amount of phase shift (possibly time varying).
Essentially this is what the Simulink Test toolbox’s ‘Baseline Tests’ feature can do, see e.g. this figure from their docs:
To me, it looks like that is simply testing whether the observed line is \pm a constant tolerance from the baseline, i.e. whether |observed - baseline| < tolerance. I’m not sure I see anything more complicated in those plots.
If by phase shift you mean something like |observed(t) - baseline(t-1)| < tol, seems like that would probably be fairly straightforward to accommodate as well.
In situations like these, I sometimes make use of Dynamic Time Warping (DTW) to allow for a slight warping of the series to fit each other. Whether or not that is reasonable of course depends a lot of the specifics of your use case. I also have a slightly higher-level package for aligning signals that are sometimes useful for comparing signals SignalAlignment.jl
