Using DiffEqFlux.jl for system identification with partially known system dynamics

New to Julia
1

Hi everyone. I want to use the DiffEqFlux.jl package to model a system of interest. I have access to a dataset which contains measurements of all system states as well as the external control inputs that act on it. A first order model of some of the states is also known. Can this knowledge be incorporated into the NeuralODE framework talked about here ?

I am new to Julia having previously programmed in Python and MATLAB. What would be the requisite background in Julia needed to be able to achieve the above?

Thanks

2

The missing physics example in the SciML docs does pretty much what you want, though it does not use NeuralODE specifically.

1 Like
3

Indeed I would recommend using this route of SciMLSensitivity.jl directly instead of DiffEqFlux.jl as it will be a bit more flexible than the full NeuralODE form. DiffEqFlux is really about pre-made layers.

1 Like
4

Hi @ChrisRackauckas @albheim Thank you both for your very helpful suggestions. This is a good starting point. I played around with this example a little and realized that the trained network is very sensitive to u0 (which was set by the random seed). If I changed the random seed, and compared solution from the ODE and the predicted solution using the pre-trained model, the UDE approximation was way off and even diverged. Is there a way in which I can train this network on multiple trajectories to rule out overfitting ?

5

Yeah that’s the next tutorial:

https://docs.sciml.ai/Overview/stable/showcase/bayesian_neural_ode/

There’s also a bunch of other tutorials on making the fitting more stable, like:

and multiple shooting:

I’m never really sure how many details to put into a first tutorial :sweat_smile:

1 Like
6

@ChrisRackauckas Awesome. I’ll check these out. Btw, these tutorials are great. Thank you for your effort !

1 Like
7

@ChrisRackauckas I did try out the multiple shooting technique using the tutorial you linked. The performance was much better compared to just training on the full trajectory once as in the missing physics example. Also, found out that ReLu worked better than rbf. I still don’t fully understand how Bayesian Neural ODE would apply here? Multiple shooting is great but I want to be able to train over other trajectories too. The reason is that in any given trajectory the control inputs have fairly low variation. It is among the different trajectories that control input varies. Any suggestions ?

8

Hi, Thanks guys for posting the above information.

My use-case is based on a discrete time space equation. I was following the
missing_physics which solves for a continuous time space using ODEProblem solver.
I tried adapting the missing_physics blog to my problem using Discrete_solver.
the training runs with the ADAM optimizer for 3000 iterations, (However could not reduce loss much and its a bad learner) and BFGS does not run saying instability detected.

Is there a solution for this? Or I am on incorrect path using Discrete solve.
I could still envy for more simpler approach adapting LUX in approximating missing term. Just like we can solve discrete time equation with “for loop”.

9

Using a straight loop with a neural network might be the simplest way to do there. There’s also GitHub - SciML/DifferenceEquations.jl: Solving difference equations with DifferenceEquations.jl and the SciML ecosystem. for if your model is a state space model. What have you tried so far?