Flux and automatic differentiation

Hello all,

This may be a naive question: how does Flux practice automatic differentiation? (i.e. according to which principle; I am aware we are dealing with Zygote)
Where can I possibly find “educational” documentation for this particular aspect? Please note that I am not a mathematician.

Thank in advance,
T.

I found something which is extremely helpful here: GitHub - MikeInnes/diff-zoo: Differentiation for Hackers :grinning:

2 Likes

See the manual,

https://fluxml.ai/Zygote.jl/dev/internals/

and the paper linked there.

I am very grateful to you, Tamas Papp, for providing me with the link to “What Zygote does”.
I am currently working on the diff-zoo written by Mike Innes. However, I have already browsed the page you mention. I rather like the last sentence:

Of course, our goal is that you never have to do this, but until Zygote is more mature it can be a useful way to narrow down test cases. :wink: