Speed up compilation with Zygote and PINN

[de-souza]

Hi,

Based on what was said in a recent topic, I would recommend the approach taken by TaylorDiff.jl for higher order automatic differentiation. See the discussion and links in that topic:

Is it possible to do Nested AD ~elegantly~ in Julia? (PINNs)

To differentiate a loss function defined in terms of a network’s derivatives seems to have been an issue since forever, specially when Zygote is involved; see [1] [2] [3] and many more over at Zygote’s git. In many of these threads (specially pre 2022) it is said that the release of Diffractor.jl would correct this issue. Now that it seems that Diffractor is mostly dead, what is left? In my case, because I am working with exotic architectures, it is not possible to use @ChrisRackauckas’s Neura…