Neural ODE with irregular Data Observations?

Specific Domains Machine Learning
1

Hi

I have a dataset where the observed data is measured at irregular timesteps. I would like to use a neural ODE to interpolate and forecast.

At the moment, I am using a feed-forward NN with fairly good success, however the time dimension makes me think a neural ODE may help when forecasting.

Are there any ‘gotchas’ when handling irregular time steps, or would I need to look at a different architecture altogether?

Thanks.

1 Like
2

I think NeuralODEs are a good fit when dealing with irregular time steps, nothing in the architecture enforce regular time steps.
Whereas RNN or other classic architecture on the opposite requires regular time steps.

In DiffEqFlux.jl you would need to saveat your time steps when computing the loss.

As for gotchas, if you use MSE for the loss, you might overweight the importance of a region where you have a lot of points compare to region with less data. One possible fix would be to consider a loss with an integral of the MSE (that you could estimate using Integrals.jl). (Note that for regular time steps this integral loss should be very much like the usual MSE).

2 Likes
3

Thank you for your reply! I am going to do some testing over on my problem but it sounds doable now.