To differentiate a loss function defined in terms of a network’s derivatives seems to have been an issue since forever, specially when Zygote is involved; see [1] [2] [3] and many more over at Zygote’s git. In many of these threads (specially pre 2022) it is said that the release of Diffractor.jl w…

An update for those who find this thread in the future: It is not advised to implement PINNs this way in Julia. The code posted above runs, but the gradients are incorrect (by a very small amount). Presently, there seems to be no AD library in Julia that can do (reverse) nested differentiation of c…

Is it possible to do Nested AD ~elegantly~ in Julia? (PINNs)

General Usage

ChrisRackauckas May 16, 2023, 1:40pm 4

Nested AD for this kind of thing is asymptotically much worse than numerical. If you’re going to use anything else, the thing to try is:

Topic		Replies	Views
Higher order derivatives/ automatic differentiation Machine Learning question , flux , autodiff	5	732	December 26, 2023
[blog post] Implement your own AD with Julia in ONE day Community blog-post	33	4377	November 3, 2018
Flux differentiation error Machine Learning zygote	19	1735	November 19, 2020
Comparison of automatic differentiation tools from 2016 still accurate? Numerics differentiation	41	5940	August 16, 2018
State of automatic differentiation in Julia Machine Learning	57	22023	September 8, 2021

Is it possible to do Nested AD ~elegantly~ in Julia? (PINNs)

Related topics