Examples of dynamical physical systems with exogenous inputs described by UDEs

[Emil_Martinsen]

Can anyone point me to some good examples (code, papers, tutorials) of a dynamical physical system that are modelled with UDEs, i.e., ODEs where parts are known and other parts are unknown and modelled by a neural network, i.e. on the form:

\frac{dx}{dt} = f_{known}(t,x(t), u(t),p) + NN(t,x(t),u(t),p),

where x(t) is the state(s) of the system and u(t) is the time-varying exogenous inputs.

The UDEs could, e.g., describe mass or energy balances.

I have looked alot into these two examples:

• Lotka-Volterra UDE (no exogenous inputs, simulated data):
  Automatically Discover Missing Physics by Embedding Machine Learning into Differential Equations · Overview of Julia's SciML
• Hammerstein system w. exogenous inputs (rhs is pure NN):
  Handling Exogenous Input Signals · SciMLSensitivity.jl

But other than that, it is difficult to find good examples to get inspiration from. Maybe I am just looking the wrong places.

Thx!

[ChrisRackauckas]

@Emil_Martinsen

where x(t) is the state(s) of the system and u(t) is the time-varying exogenous inputs.

Not sure if there’s a good example online of this exact form right now? I mean you’d just use DataInterpolations.jl with it, so there’s not much to it. @SebastianM-C do you have a good example to point to?

[Emil_Martinsen]

Hi Chris, thx for your reply. Fair. Yes I use DataInterpolations.jl for the inputs. My main challenge is that the gradient computation of the loss functions seems to be extremely slow with my setup. It seems like something is wrong or done in a inefficient way.

So I made a new post with a MWE of my setup - maybe you or someone else can see what is missing.

Link to post:

AD of UDEs - how to set it up?

Hi! Im new to Julia and SciML but I am really curious to see how combining ODE’s with neural network could improve the predictive performance of the model we are developing for solvent-based CO2 capture. My current setup includes dynamic states, multiple independent timeseries of real data exogenous inputs, unknown dynamics modelled as a neural network (NN). I use DataInterpolations.jl to handle the inputs. I want to train the NN by minimising a loss function (MSE across all the independent…

[baggepinnen]

I have a small example here

SciML: Adaptive Universal Differential Equation · LowLevelParticleFilters...

it learns the network weights online, i.e., no separate training and using phases.

There’s another example here from an ongoing pet project of mine

SciML: Learning a sunshine disturbance model · LowLevelParticleFilters...

it uses a slightly simpler function approximator than an NN though.

[SebastianM-C]

I also have an example with Experimental Design for Missing Physics - ScienceDirect

This is also showcased in Optimal Data Gathering for Missing Physics · Overview of Julia's SciML