BTW, I’ll like to see what neural network you come up with to fit Lotka-Volterra. I was running the animations and recording them live on a core i5 laptop, so I kept it to the simple case . But when I did try to train LV with one hidden layer the NN didn’t seem big enough to capture the function. But on my laptop I couldn’t use the GPUs, so I’m interested to see what kind of NN can be used here .
(Also, there’s a much better way to train this, but that’s the topic for another publication)
Hi, thanks for the help, it works now. I cannot use a GPU either, simply because I don’t have one. So far my experience is that these networks are difficult to train. I don’t think it is the size or depth of the network. I think it is because of the nature of ODEs. Perturbations are amplified exponentially in time and that is hard to handle with any optimisation. Anyway, I will do some more experimentation before making a judgement.
My strategy would be to train with many short trajectories first and then improve on that with smaller number of longer trajectories. At the moment I have no clue how to do multiple trajectories, my modification of the loss function does not work. If you can give an example with two trajectories, that would be great. Thanks
That’s multiple shooting. We actually do that in DiffEq-proper: http://docs.juliadiffeq.org/latest/analysis/parameter_estimation.html. We will be putting a paper out on how to loss functions that improve the fitting. What the blog post shows is the training using single shooting which is what the paper shows, but we know that there are better ways .