Auto-diff Friendly GPU Stencils

You could try things like @tullio y[i] := -x[i] + 2x[i+1] - x[i+2], I believe such cases ought to be fairly efficient (including GPU & derivatives).

(You may also be interested in ParallelStencil.jl but not sure this will help with derivatives.)