Differential Equations and Parameter Estimation


I’m about to attempt parameter estimation using DifferentialEquations, and have a couple of questions – I hope someone can share their experience. I’ve read the tutorial, and some previous discussion.

  • My problem is a single PDE of how temperature in a tank varies with some boundary inputs (influent temperature, some heating power, some volumetric flow rates, some control inputs). I have discretized the PDE using the finite difference method.
  • I have data consisting of 5 inputs (“manipulated variables”) and 2 temperatures outputs sampled every minute.
  • One of the sampled output temperatures T_\mathrm{s} is a combination of an influent temperature T_\mathrm{i} and the (unknown) effluent temperature T_\mathrm{e}; T_\mathrm{s} = u_\mathrm{v} T_\mathrm{e} + (1-u_\mathrm{v})T_\mathrm{i} where u_\mathrm{v} is a control input (a split range valve setting).
  • Because of the form of measurement/data T_\mathrm{s} is not a state, I assume that it is best to pose the model as a DAE so that I can have T_\mathrm{s} as a solution – ODEs can only have states as a solution, I guess.
  • I will probably use 2 hyper parameters in the model fitting: (i) the number n of compartments in the discretized model, and (ii) the weight \lambda of the regularization term.
  • I noted Chris’ response to alewolf on how to choose which variables in the DAE solver are used in the loss function (save_idxs argument).

My questions are the following…

  1. My input data are available at a discrete time interval (1 min). I can handle this in two ways:
    a. Set up a for loop, and solve the DAE over 1 minute with constant input, passing the final result as initial time to the next iteration in the for loop, and let the cost function be the sum of each sub sum, or
    b. Create an interpolation function for the input data so that the DAE solver knows the value at all times.
    Which is recommended, (a) or (b)?
  2. In general, initial states will be unknown, and I assume that I should pose these as unknown parameters in the model fitting? (n compartments…)
  3. The regularization term – it looks like the standard is to add the squared 2-norm of the parameter vector. If I have some idea of what the parameters should be… which is best?
    a. Add bounds for the parameters in the optimization problem, or
    b. Formulate the regularization term to be the norm of the deviation of the parameter from some initial guess?
    c. Combining the above, i.e., combine (a) and (b)?
  4. If my validation set is not immediately following in time my training set, the initial states of the validation set will be unknown. How to I handle this problem?
  • Use state estimation for the validation problem?
  • Use state estimation for both the validation problem and the training problem for assessing hyper parameters?
  1. If I vary my two hyper parameters simultaneously (n and \lambda) and compare the RMSE of both the training set and the validation set to choose the hyper parameters, I get a 2D problem. Should I tackle one at a time?

the tank is agitated? you can have some boundaries using a CSTR and a PFR results as limits


Neither. Just solve one DAE with saveat=1, and then use the result array against all data points at once.

Yes, just make them functions of parameters being fit

(a) is what’s usually done, (b) is a little weird but is in some sense related to putting a Gaussian prior distribution and doing MAP.

Your choice.

Or use the gradient.

1 Like

The tank model describes stratification in the temperature, so the temperature is distributed.


Thanks, Chris. My first question was perhaps not very clear. My model is typically:

  • \frac{dx}{dt} = f(x,u;\theta), with sampled outputs from y=g(x,u;\theta).
  • I have input-output pairs (u_i,y_i),\quad i\in \{1,\ldots,N_d\}, thus N_d data points sampled at 1 min interval.
  • The model solution x(t) is influenced by the input u(t)\in \mathbb{R}^5, and so is the output y(t)\in \mathbb{R}^2.
  • Because I don’t observe the states directly, but instead the mapped y variable, I think I need a DAE formulation instead of an ODE formulation.
  • Critical element of question: Because u(t) influences the solution, I assume that I need to either…
    i. Make an interpolant for u_i so that the DAE has u(t) available, or
    ii. Make a for loop for solving the model in the interval [t_i,t_{i+1}) with constant input u_i, and then let the solution x(t_{i+1}) become the initial value for the next value of i??

Hm… there is perhaps a third way that looks like what you suggest:

  • Make the input an unknown parameter, use both u_i and y_i as data, and use parameter fitting to find the value of the unknown input.
  • In this approach, it should be straightforward to find the unknown input parameter because it will be equal to the datapoints u_i??
  • I would still need to do some interpolation-like thing in the DAE model, to let the model know which of the input parameters to use.
  • I have lots of data; N_d is in the order of 1e4 – possibly sampled too often.

Anyway, is there an example around on model fitting with models influenced by inputs?


Only if they are implicitly defined. Otherwise it’s still an ODE. I guess the API on the parameter estimation is the issue here. You can define your own cost function to that uses explicit definitions from the ODE.

This shows up enough that we made https://github.com/UMCTM/DataInterpolations.jl for this.

1 Like