# ForwardDiff.jl with FFTW.jl

**URL:** <https://discourse.julialang.org/t/forwarddiff-jl-with-fftw-jl/97366>\
**Category:** Numerics\
**Created:** [April 11, 2023, 9:47pm UTC](https://discourse.julialang.org/t/forwarddiff-jl-with-fftw-jl/97366 "2023-04-11T21:47:54Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Bill\_Holmes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bill_holmes/32/47921_2.png) [@Bill\_Holmes](https://discourse.julialang.org/u/Bill_Holmes)\
**Post date:** [April 11, 2023, 9:47pm UTC](https://discourse.julialang.org/t/forwarddiff-jl-with-fftw-jl/97366/1 "2023-04-11T21:47:55Z")

</div>

Hello all. I am working with ForwardDiff.jl on a problem that requires computing kernel density estimates (and thus convolutions which boils down to FFT). I am searching for a way to do this using autodifferentation. I realize there are a number of AD frameworks. My problems are relatively parametrically small (order of 100 params), involves control flow (while loops and if statements), and requires mutation. Based on this, I’ve narrowed down on ForwardDiff (though looking into Enzyme).

The problem is essentially of the following form. Given some parameters (p), use a black box (for our purposes) to draw samples from a distribution parameterized by p - Dist(p), use kernel density estimation to construct an approximate pdf at known data points pdf(Dist(p),Data). I have verified that my black box that I use to draw samples is ForwardDiff compatible. However the kernel density estimator is the problem. The most efficient way to compute a kernel density estimate is to construct a noisy histogram (H) from samples, and then smooth against a kernel (k). This takes the form

f(x;p) = H(y;p) \star k(y) (x) .

The most efficient way to do this is to FFT everything, turn the convolution into multiplication, then ifft. I currently use ‘rfft’ and ‘irfft’ for all this from FFTW.jl since I’m working exclusively with real valued data.

The problem becomes that eventually the full program differentiation needs the f\_p derivative, which requires differentiating the FFT. Unfortunately FFTW is not ForwardDiff compatible (does not have appropriate typing to accept duals). Mathematically the derivative is trivial

\frac{d \, FFT[f](w;p)}{dp} = FFT \left[\frac{df}{dp} (x;p) \right](w)

The question is, how do you get ForwardDiff to work with this? Or alternatively, is there an efficient implementation of convolutions that would be compliant? That after all, is what I need (but is O(n^2) instead of O(n\*log(n))).

I did find the following post. However it involved complex FFTs, FFT plans, and the solution was beyond my ability to follow.

> [@ForwardDiff and Zygote cannot automatically differentiate (AD) function from C^n to R that uses FFT](https://discourse.julialang.org/t/forwarddiff-and-zygote-cannot-automatically-differentiate-ad-function-from-c-n-to-r-that-uses-fft/52440):
>
> I’ve encountered so many bugs using ForwardDiff and Zygote to do automatic differentiation for my objective function that takes complex vector as input and outputs a real number. I don’t think I have the expertise to debug myself so I’m posting my problem here. ForwardDiff is just bad and a quick search suggested that it doesn’t support complex stuff as well as Zygote. For Zygote, the most common error I got is about how plan\_fft doesn’t have a method for the ForwardDiff AD datatype: MethodErro…

---

<div class="post-metadata">

**Author:** ![dlfivefifty](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlfivefifty/32/1959_2.png) [@dlfivefifty](https://discourse.julialang.org/u/dlfivefifty)\
**Post date:** [April 11, 2023, 11:24pm UTC](https://discourse.julialang.org/t/forwarddiff-jl-with-fftw-jl/97366/2 "2023-04-11T23:24:14Z")

</div>

> **[GitHub - JuliaApproximation/FastTransformsForwardDiff.jl: A Julia package to...](https://github.com/JuliaApproximation/FastTransformsForwardDiff.jl)**
>
> A Julia package to support forward-mode auto-differentiation for fast transforms - GitHub - JuliaApproximation/FastTransformsForwardDiff.jl: A Julia package to support forward-mode auto-differentia...

---

<div class="post-metadata">

**Author:** ![Bill\_Holmes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bill_holmes/32/47921_2.png) [@Bill\_Holmes](https://discourse.julialang.org/u/Bill_Holmes)\
**Post date:** [April 12, 2023, 12:07am UTC](https://discourse.julialang.org/t/forwarddiff-jl-with-fftw-jl/97366/3 "2023-04-12T00:07:51Z")

</div>

Thanks for the response. Is there any examples or docs for how this would integrate with a broader ForwardDiff package? FFT is usually part of a larger computation to which AD is being applied.

---

<div class="post-metadata">

**Author:** ![dlfivefifty](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlfivefifty/32/1959_2.png) [@dlfivefifty](https://discourse.julialang.org/u/dlfivefifty)\
**Post date:** [April 12, 2023, 12:10am UTC](https://discourse.julialang.org/t/forwarddiff-jl-with-fftw-jl/97366/4 "2023-04-12T00:10:13Z")

</div>

I’m pretty sure you load both packages and it just works…not sure what there is to document.

If it doesn’t work file a GitHub issue

---

<div class="post-metadata">

**Author:** ![tom-plaa](https://avatars.discourse-cdn.com/v4/letter/t/d26b3c/32.png) [@tom-plaa](https://discourse.julialang.org/u/tom-plaa)\
**Post date:** [September 23, 2025, 1:26pm UTC](https://discourse.julialang.org/t/forwarddiff-jl-with-fftw-jl/97366/5 "2025-09-23T13:26:40Z")

</div>

Sorry for reopening this after such a long time, but what is the current recommended setup for this?  
I have a personal package that is relying on `FastTransformsForwardDiff` which is locked to pre-1.0 `ForwardDiff` (still on 0.10), and I don’t know if I should just keep it on that version or if there is another way to get FFT’s working with `ForwardDiff`.

EDIT: As an alternative hack, and not really understanding what the breaking changes in `ForwardDiff` \>= 1.0 are, would copying the code in these pull requests work as a “quick fix”?

> <https://github.com/JuliaMath/AbstractFFTs.jl/pull/138>
>
> This is moving type-piracy code from https://github.com/JuliaApproximation/FastT…ransformsForwardDiff.jl to an extension here.

> <https://github.com/JuliaMath/FFTW.jl/pull/311>
>
> This moves over code from FastTransformsForwardDiff.jl, and goes with https://gi…thub.com/JuliaMath/AbstractFFTs.jl/pull/138

Or in the case I’m using `GenericFFT` instead of `FFTW`, just the first merge request on `AbstractFFTs` without the one on `FFTW`?

---

<div class="post-metadata">

**Author:** ![dlfivefifty](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlfivefifty/32/1959_2.png) [@dlfivefifty](https://discourse.julialang.org/u/dlfivefifty)\
**Post date:** [November 14, 2025, 8:15am UTC](https://discourse.julialang.org/t/forwarddiff-jl-with-fftw-jl/97366/6 "2025-11-14T08:15:54Z")

</div>

> [@tom-plaa](#):
>
> `FastTransformsForwardDiff`

Should be updated now