Concept of continuous depths for neural networks in UDEs

Specific Domains Modelling & Simulations
1

I recently came across the concept of continuous depth in Neural ODEs. I was wondering if the neural networks in the UDE framework have a similar concept of continuous depths. @ChrisRackauckas

2

Yes, ODEs are continuous in their independent variable.

3

@ChrisRackauckas Sorry. I think I did not frame my question properly. Is the depth of the neural network considered a ‘parameter’ in the UDE framework? Is it similar to continuous depth of neural ODEs?

4

I mean, you can think about it like that but I don’t think it’s a very useful abstraction if you’re thinking about real models.

5

Yeah, you have to be careful that with continuous depth you can reparametrize depth to make a network of depth 10 into a network of depth 3.1415 without changing it…

6

Thanks

7

@ChrisRackauckas @mschauer Is there any benefit in terms of training accuracy by parameterizing the depth compared to a case where it is not done so?

8

I don’t think that framework of thinking even makes sense with UDEs. Take a look at this recent talk:

It’s all about mechanistic models. If you’re not talking mechanistic models, then you’re not talking UDEs.

1 Like