Definitely NeuralPDE and physics-informed neural networks are the wrong way to do this. SciMLSensitivity.jl and using direct inference will be a few orders of magnitude faster.
Do instead prob = ODEProblem{false}(sys,init_conds,tspan,init_params) to enforce the out-of-place function build.