Parallel computing and GPU support in neuralPDE.jl package

[ChrisRackauckas]

@Aakhash_Sundaresan

’m using 9 neural networks with 3 Hidden layers and 40 neurons in each layer. So how would I do the matmul smoke test for this case?

Test 40xN matmuls. N would be the batch size in the sampling, or number of samples. So like, 40x100.

@Aakhash_Sundaresan

What are the factors that affect the performance of the NeuralPDE framework? DOes it have any limit on the number of equations and BC’s that can be used, or the maximum number of layers and neurons that one can use? Or the maximum order of the differential equation that can be used?..

There is no maximum order, though physical equations don’t tend to have above 4 and I think that’s where most of the hard-coded extra optimizations would stop. There is not a limit on the number of equations, and you generally scale linearly with that. You just scale quadratically with larger size (or think cubically as the samples grow), which is a fundamental limitation because of the matmuls in the neural networks.

@Aakhash_Sundaresan

Also, is it that GridTraining is faster on the GPU’s and QuasiRandom sampling isn’t? On the documentation page, it says to use GridTraining only for testing purposes. That means GridTraining strategu cannot be used for final training or what is the exact reason

GridTraining does not overcome curse of dimensionality, does not hit random points so it tends to not give great results between grid points, and has slower convergence than a quasi-random low discrepancy sampler.