I come from the Tapenade source transformation community and use Fortran as my primary language. I recently moved to Julia and slowly adapting to the language. I have problems understanding the algorithmic differentiation in Zygote (aka differential programming).
I like to explain my problem with a simple code example. Say I want to differentiate the following function
h(x, y) = 3x^2 + 2x + 1 + y*x - y
In FORTRAN it translates to
SUBROUTINE CALC_H(x, y, h) h = 3*x*x + 2*x + 1 + y*x - y END SUBROUTINE CALC_H
in the forward mode. Most AD tool including Tapenade would return the following forward gradient code,
SUBROUTINE CALC_H_D(x, xd, y, yd, h, hd) hd = (3*2*x+y+2)*xd + (x-1.0)*yd h = 3*x*x + 2*x + 1 + y*x - y END SUBROUTINE CALC_H_D
So basically the function takes not just x,y but also increments xd, yd as arguments.
This is very useful to perform Jaobian-vector products automatically by simply
specifying xd, yd inputs. Also the output returns the gradient in hd. We have a
similar semantics for the reverse mode.
SUBROUTINE CALC_H_B(x, xb, y, yb, h, hb) xb = (6*x+y+2)*hb yb = (x-1.0)*hb hb = 0.0_8 END SUBROUTINE CALC_H_B
So I can set the value of
hd and get that value multiplied inside the ad routines.
All that I could do in Zygote for the moment is
julia> using Zygote julia> h(x, y) = 3x^2 + 2x + 1 + y*x - y h (generic function with 1 method) julia> gradient(h, 3.0, 5.0) (25.0, 2.0)
provide only two parameters
hd in the adjoint mode of Zygote
is 1.0! This is a big limitation for my work. Is there a way to achieve what I
want in Zygote. My knowledge of Julia and Zygote is very limited.
Also how do I specify
xd, yd in the forward mode?