Here is a minimal working example
const x = -10:10 f(m) = exp.(-1.5 .* (x .- m).^2) function rs(d,xs) d0 = f(xs) d .- sum(d .* d0) / sum(d0 .* d0) .* d0 end drs(d,xs) = (rs(d,xs + 1e-6) .- rs(d,xs)) / 1e-6 drs_auto(d,x) = Zygote.jacobian(rs, d,x) drs_auto_v2(d,x) = Zygote.jacobian(z -> rs(d,z),x) drs_auto_v3(d,x) = ForwardDiff.derivative(z -> rs(d,z),x)
and I get
Zygote is >50 times slower than the numerical derivative, ForwardDiff is 2 times faster.
I know that Zygote is reverse mode, but is that the reason of the performance difference? Can the Zygote call be improved?
Notice that I only need the derivatives of rs with respect to x, I do not need to compute the derivatives with respect to d. Is there any way to tell that to Zygote? And would that matter? Using the anonymous function was just my quick failed attempt.