To differentiate a loss function defined in terms of a network’s derivatives seems to have been an issue since forever, specially when Zygote is involved; see [1] [2] [3] and many more over at Zygote’s git. In many of these threads (specially pre 2022) it is said that the release of Diffractor.jl w…

An update for those who find this thread in the future: It is not advised to implement PINNs this way in Julia. The code posted above runs, but the gradients are incorrect (by a very small amount). Presently, there seems to be no AD library in Julia that can do (reverse) nested differentiation of c…

Is it possible to do Nested AD ~elegantly~ in Julia? (PINNs)

General Usage

ChrisRackauckas May 16, 2023, 1:40pm 4

Nested AD for this kind of thing is asymptotically much worse than numerical. If you’re going to use anything else, the thing to try is:

Topic		Replies	Views
Higher order derivatives/ automatic differentiation Machine Learning question , flux , autodiff	5	758	December 26, 2023
What is the best way to implement PINN in Julia New to Julia pde , neural-network , autodiff	18	831	April 6, 2025
Nested and different AD methods altogether: How to add AD calculations inside my loss function when using neural differential equations? Machine Learning sciml , ad , neural-network , differentialequation	9	1077	September 28, 2024
Flux PINN 1D Burgers Machine Learning question	11	950	May 23, 2025
Nested AD with Lux etc Machine Learning ad	26	1452	May 1, 2024

Is it possible to do Nested AD ~elegantly~ in Julia? (PINNs)

Related topics