Pushforward in Enzyme.jl

Specific Domains
1

Hello is there a way to obtain the pushforward of a function using Enzyme.jl?

Concretely, given the function foo below, I want the enzyme to produce the function dfoo:

#output in d, mutated
function foo(d,c)
    d.=c.^2
    return nothing
end
#output in d, mutated
function dfoo(dd,c,dc)
    dd.=2.0.*c.*dc
    return nothing
end

Which is the Frechet derivative at c applied on dc. I tried:

c=2*ones(3)
dc=rand(3)
d=zeros(3)
dd=zeros(3)
Enzyme.autodiff(bla,Dupl(d,dd),Duplicated(c,dc))
dd # all zeros!

which does not work, I appreciate any help :slight_smile:

2

You’re looking for forward mode AD. The default with Enzyme is to do reverse mode (which propagates derivatives from outputs to inputs, hence when dd remains zeros). Forward mode will do as you desire (propagate derivatives from inputs to outputs).

Try autodiff(Forward, foo, Duplicated(d,dd), Duplicated(c,dc))

1 Like
3

Thank you! I have one more question, does this do the right thing and compute the derivative at almost the cost of 2 applications of foo or does it scale like the dimensions of the input (size of c)?

4

Yes forward mode in Enzyme should be approximately 2 times the runtime of the original function, or faster.

1 Like