# Using DiffEqFlux.jl for system identification with partially known system dynamics

**URL:** <https://discourse.julialang.org/t/using-diffeqflux-jl-for-system-identification-with-partially-known-system-dynamics/98197>\
**Category:** New to Julia\
**Created:** [May 2, 2023, 1:09pm UTC](https://discourse.julialang.org/t/using-diffeqflux-jl-for-system-identification-with-partially-known-system-dynamics/98197 "2023-05-02T13:09:58Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![gsh19](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gsh19/32/49350_2.png) [@gsh19](https://discourse.julialang.org/u/gsh19)\
**Post date:** [May 2, 2023, 1:09pm UTC](https://discourse.julialang.org/t/using-diffeqflux-jl-for-system-identification-with-partially-known-system-dynamics/98197/1 "2023-05-02T13:09:58Z")

</div>

Hi everyone. I want to use the DiffEqFlux.jl package to model a system of interest. I have access to a dataset which contains measurements of all system states as well as the external control inputs that act on it. A first order model of some of the states is also known. Can this knowledge be incorporated into the NeuralODE framework talked about [here](https://arxiv.org/abs/2001.04385) ?

I am new to Julia having previously programmed in Python and MATLAB. What would be the requisite background in Julia needed to be able to achieve the above?

Thanks

---

<div class="post-metadata">

**Author:** ![albheim](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/albheim/32/34660_2.png) [@albheim](https://discourse.julialang.org/u/albheim)\
**Post date:** [May 2, 2023, 1:52pm UTC](https://discourse.julialang.org/t/using-diffeqflux-jl-for-system-identification-with-partially-known-system-dynamics/98197/2 "2023-05-02T13:52:46Z")

</div>

The [missing physics](https://docs.sciml.ai/Overview/stable/showcase/missing_physics/) example in the SciML docs does pretty much what you want, though it does not use NeuralODE specifically.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [May 2, 2023, 3:48pm UTC](https://discourse.julialang.org/t/using-diffeqflux-jl-for-system-identification-with-partially-known-system-dynamics/98197/3 "2023-05-02T15:48:18Z")

</div>

Indeed I would recommend using this route of SciMLSensitivity.jl directly instead of DiffEqFlux.jl as it will be a bit more flexible than the full NeuralODE form. DiffEqFlux is really about pre-made layers.

---

<div class="post-metadata">

**Author:** ![gsh19](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gsh19/32/49350_2.png) [@gsh19](https://discourse.julialang.org/u/gsh19)\
**Post date:** [May 4, 2023, 4:20pm UTC](https://discourse.julialang.org/t/using-diffeqflux-jl-for-system-identification-with-partially-known-system-dynamics/98197/4 "2023-05-04T16:20:20Z")

</div>

Hi @ChrisRackauckas @albheim Thank you both for your very helpful suggestions. This is a good starting point. I played around with this example a little and realized that the trained network is very sensitive to u0 (which was set by the random seed). If I changed the random seed, and compared solution from the ODE and the predicted solution using the pre-trained model, the UDE approximation was way off and even diverged. Is there a way in which I can train this network on multiple trajectories to rule out overfitting ?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [May 4, 2023, 4:22pm UTC](https://discourse.julialang.org/t/using-diffeqflux-jl-for-system-identification-with-partially-known-system-dynamics/98197/5 "2023-05-04T16:22:44Z")

</div>

> [@gsh19](#):
>
> Is there a way in which I can train this network on multiple trajectories to rule out overfitting ?

Yeah that’s the next tutorial:

[https://docs.sciml.ai/Overview/stable/showcase/bayesian\_neural\_ode/](https://docs.sciml.ai/Overview/stable/showcase/bayesian_neural_ode/)

There’s also a bunch of other tutorials on making the fitting more stable, like:

> **[Strategies to Avoid Local Minima · SciMLSensitivity.jl](https://docs.sciml.ai/SciMLSensitivity/dev/tutorials/training_tips/local_minima/)**
>
> Documentation for SciMLSensitivity.jl.

and multiple shooting:

> **[Multiple Shooting · DiffEqFlux.jl](https://docs.sciml.ai/DiffEqFlux/dev/examples/multiple_shooting/)**
>
> Documentation for DiffEqFlux.jl.

I’m never really sure how many details to put into a first tutorial 😅

---

<div class="post-metadata">

**Author:** ![gsh19](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gsh19/32/49350_2.png) [@gsh19](https://discourse.julialang.org/u/gsh19)\
**Post date:** [May 4, 2023, 4:31pm UTC](https://discourse.julialang.org/t/using-diffeqflux-jl-for-system-identification-with-partially-known-system-dynamics/98197/6 "2023-05-04T16:31:39Z")

</div>

@ChrisRackauckas Awesome. I’ll check these out. Btw, these tutorials are great. Thank you for your effort !

---

<div class="post-metadata">

**Author:** ![gsh19](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gsh19/32/49350_2.png) [@gsh19](https://discourse.julialang.org/u/gsh19)\
**Post date:** [May 8, 2023, 2:36pm UTC](https://discourse.julialang.org/t/using-diffeqflux-jl-for-system-identification-with-partially-known-system-dynamics/98197/7 "2023-05-08T14:36:56Z")

</div>

@ChrisRackauckas I did try out the multiple shooting technique using the tutorial you linked. The performance was much better compared to just training on the full trajectory once as in the missing physics example. Also, found out that ReLu worked better than rbf. I still don’t fully understand how Bayesian Neural ODE would apply here? Multiple shooting is great but I want to be able to train over other trajectories too. The reason is that in any given trajectory the control inputs have fairly low variation. It is among the different trajectories that control input varies. Any suggestions ?

---

<div class="post-metadata">

**Author:** ![kunal](https://avatars.discourse-cdn.com/v4/letter/k/dec6dc/32.png) [@kunal](https://discourse.julialang.org/u/kunal)\
**Post date:** [September 3, 2023, 1:17am UTC](https://discourse.julialang.org/t/using-diffeqflux-jl-for-system-identification-with-partially-known-system-dynamics/98197/8 "2023-09-03T01:17:29Z")

</div>

Hi, Thanks guys for posting the above information.

My use-case is based on a discrete time space equation. I was following the  
[missing\_physics](https://docs.sciml.ai/Overview/stable/showcase/missing_physics/) which solves for a continuous time space using ODEProblem solver.  
I tried adapting the missing\_physics blog to my problem using [Discrete\_solver](https://docs.sciml.ai/DiffEqDocs/latest/solvers/discrete_solve/).  
the training runs with the ADAM optimizer for 3000 iterations, (However could not reduce loss much and its a bad learner) and BFGS does not run saying instability detected.

Is there a solution for this? Or I am on incorrect path using Discrete solve.  
I could still envy for more simpler approach adapting LUX in approximating missing term. Just like we can solve discrete time equation with “for loop”.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [September 12, 2023, 5:51pm UTC](https://discourse.julialang.org/t/using-diffeqflux-jl-for-system-identification-with-partially-known-system-dynamics/98197/9 "2023-09-12T17:51:03Z")

</div>

> [@kunal](#):
>
> Is there a solution for this? Or I am on incorrect path using Discrete solve.  
> I could still envy for more simpler approach adapting LUX in approximating missing term. Just like we can solve discrete time equation with “for loop”.

Using a straight loop with a neural network might be the simplest way to do there. There’s also [GitHub - SciML/DifferenceEquations.jl: Solving difference equations with DifferenceEquations.jl and the SciML ecosystem.](https://github.com/SciML/DifferenceEquations.jl) for if your model is a state space model. What have you tried so far?