Hello everyone,
I would like to know if there is a package on Julia adapted to Neural ODEs with time dependent parameters, or if there is a practical example of code use to allow evolution of the neural network parameters.
The form of the problem is: \frac{dz(t)}{dt} = f(z(t), \theta (t),t)
Where θ(t) are parameters that depend on time t.
Similar to https://diffeqflux.sciml.ai/dev/examples/exogenous_input/ ? Or just embedding an interpolation?
I don’t think my problem has anything to do with exogenous input signals. I should have explained myself more clearly, I would like to fit a time serie of raw data using for example the following equation as a neural ode:
\frac{d^2y}{dt^2} +a(t)y=0
the examples I find are all with independent time parameters of the form \frac{d^2y}{dt^2} +a.y=0.
for example if the dt of my solver is equal to 1 millisecond and I wanted to solve the equation on a time duration of 1 second (0.0,1.0), I will need 1000 points, so I will initialize a(t) as following:
A = rand(1000)
a = t → A[Int(floor(1000*t))]
the goal is that after each iteration of my ML model a(t) (the time serie) changes so that the output better fits my raw data.
You can just define an interpolation function. I.e., a(t) is still defined by fixed parameters, a(t,p), where now p is the set of time series values that is used to build the interpolation. You can then use GitHub - PumasAI/DataInterpolations.jl: A library of data interpolation and smoothing functions to generate a(t,p) from say a cubic spline inside of the differential equation as a function of the fixed p, and then it will differentiate w.r.t. those fixed value points. https://github.com/SciML/DiffEqFlux.jl/issues/408 has some examples of this. Let me know if you need more pointers. This would be great to make into a documentation example.
Thank you @ChrisRackauckas for your answers, i ll try to understand, then follow these steps. I will come back to you if I encounter a big obstacle or question. Finally, If I find something interesting, I will surely post it here to serve as a version 0 of a possible documentation.
Hi @ChrisRackauckas, I have tested the solution you propose and it works very well for functions a(t) defined by fixed parameters. However, in my research case, the function a(t) is much more complicated to define and I can’t get a parametric analytical form.
I came across an article on this subject. If a(t) verifies another differential equation, we can use it as a second neural ode that will be coupled to the initial neural ode.
What do you think of this method?
Here is one of the articles that explains this method:
Could you use a NN to parameterize \theta(t)? That way the parameters of the NN are fixed through time and the output of the NN are the parameters at each timepoint: \theta_t = NN(t).
https://diffeqflux.sciml.ai/dev/examples/optimal_control/
Is it possible you could post your code here for when a(t) is defined by fixed parameters? It might be useful information to others looking for solve this problem (including me).