saveat takes as large steps as possible and then interpolates, so post interpolation won’t help here. One thing that could help is to add save_idxs=1, to tell it only save the timeseries of the first variable.
But secondly, I agree with @stevengj here that you probably want to do is take random subsets from your training dataset, i.e. “minibatching”, and evaluate the cost on those subsets. Normally when the data is too dense it isn’t helpful.
But thirdly, if your dataset is that dense, then you probably want to fit it in a completely different way. You can instead interpolate your data and calculate the approximate derivatives u'(t) from cubic spline approximations. You can then write your cost/likelihoods as ||u' - f(u(t),p,t)||. This doesn’t require running differential equation solves at all and is a lot cheaper. You can take a look at how https://docs.sciml.ai/latest/analysis/parameter_estimation/#Two-Stage-method-(Non-Parametric-Collocation)-1 is implemented for one way to do the interpolation, or just use something like https://github.com/kbarbary/Dierckx.jl