Enzyme.jl Critical Issues with Basic Arithmetic Operations in Automatic Differentiation

[gdalle]

To get your mind around the way forward and reverse modes work in Enzyme, this discussion might help:

Do I understand Enzyme properly?

I think I finally have a grasp on how Enzyme works, but I would like an expert to validate this. Consider a possibly mutating vector function f(x, y) which outputs a scalar value z. We denote by x_a and y_a the contents of x and y after execution. Enzyme works with a Jacobian including all of these variables (defined by blocks): J = \begin{pmatrix} \partial x_a / \partial x & \partial x_a / \partial y \\ \partial y_a / \partial x & \partial y_a / \partial y \\ \partial z / \partial x & \partia…