# Forward Automatic Differentiation & FFTs

**URL:** <https://discourse.julialang.org/t/forward-automatic-differentiation-ffts/126036>\
**Category:** Numerics\
**Tags:** forwarddiff, autodiff, fft\
**Created:** [February 18, 2025, 4:21pm UTC](https://discourse.julialang.org/t/forward-automatic-differentiation-ffts/126036 "2025-02-18T16:21:44Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![marcobonici](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcobonici/32/20549_2.png) [@marcobonici](https://discourse.julialang.org/u/marcobonici)\
**Post date:** [February 18, 2025, 4:21pm UTC](https://discourse.julialang.org/t/forward-automatic-differentiation-ffts/126036/1 "2025-02-18T16:21:44Z")

</div>

I need to understand if there is a combination of packages that is able to wdo what I need.  
I need to do some FFTs, differentiate through them both in Forward and Backward mode with one of the Julia AD engines (it might be possible to use an engine for Forward and one for backward mode) and use into Turing.jl for some bayesian chains.  
Backward mode is not really a problem…but what about Forward? Some of my problems have parameter sizes, and hence I am fine with backward mode, but I still would like to be able to use Forward mode for some smaller problems.  
So, the list of desiderata is:

1. An FFT package that works nicely with…
2. …a Forward AD engine, which work nicely with Turing (hence ADTypes?)

Any suggestion? In this [discussion](https://discourse.julialang.org/t/forwarddiff-jl-with-fftw-jl/97366) a solution was proposed, but I’d like to have something not relying on a package that has not been actively mantained for years (and likely does not work with Turing.jl).  
Is there a working solution to my problem? Or should I just give up on Forward mode AD?

---

<div class="post-metadata">

**Author:** ![dlfivefifty](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlfivefifty/32/1959_2.png) [@dlfivefifty](https://discourse.julialang.org/u/dlfivefifty)\
**Post date:** [February 18, 2025, 4:28pm UTC](https://discourse.julialang.org/t/forward-automatic-differentiation-ffts/126036/2 "2025-02-18T16:28:35Z")

</div>

There’s pull requests to support ForwardDiff.jl, try on these branches:

> <https://github.com/JuliaMath/AbstractFFTs.jl/pull/138>
>
> This is moving type-piracy code from https://github.com/JuliaApproximation/FastT…ransformsForwardDiff.jl to an extension here.

> <https://github.com/JuliaMath/FFTW.jl/pull/311>
>
> This moves over code from FastTransformsForwardDiff.jl, and goes with https://gi…thub.com/JuliaMath/AbstractFFTs.jl/pull/138

---

<div class="post-metadata">

**Author:** ![marcobonici](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcobonici/32/20549_2.png) [@marcobonici](https://discourse.julialang.org/u/marcobonici)\
**Post date:** [February 18, 2025, 4:35pm UTC](https://discourse.julialang.org/t/forward-automatic-differentiation-ffts/126036/3 "2025-02-18T16:35:29Z")

</div>

Thanks! I’ll try those today and I’ll report back…in the meantime, thanks!

If it works, do you have a rough estimate of the timeline for the merging?  
Thanks again!

---

<div class="post-metadata">

**Author:** ![dlfivefifty](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlfivefifty/32/1959_2.png) [@dlfivefifty](https://discourse.julialang.org/u/dlfivefifty)\
**Post date:** [February 18, 2025, 4:42pm UTC](https://discourse.julialang.org/t/forward-automatic-differentiation-ffts/126036/4 "2025-02-18T16:42:25Z")

</div>

I’m busy lecturing and other things right now… I imagine it will take a lot of back-and-forth with maintainers. So unfortunately unless you or someone else takes the lead, probably not anytime soon.

---

<div class="post-metadata">

**Author:** ![marcobonici](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcobonici/32/20549_2.png) [@marcobonici](https://discourse.julialang.org/u/marcobonici)\
**Post date:** [February 18, 2025, 5:20pm UTC](https://discourse.julialang.org/t/forward-automatic-differentiation-ffts/126036/5 "2025-02-18T17:20:23Z")

</div>

If it works, I would be interested.  
I might ask some guidance, but I think this will also come from the interactions with the mantainers.  
I gave a quick look into the AbstractFFTs package. Which are the big chunks missing? Some unit tests for a few lines, is this right? (Beside what the mantainers might ask).

---

<div class="post-metadata">

**Author:** ![dlfivefifty](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlfivefifty/32/1959_2.png) [@dlfivefifty](https://discourse.julialang.org/u/dlfivefifty)\
**Post date:** [February 18, 2025, 6:34pm UTC](https://discourse.julialang.org/t/forward-automatic-differentiation-ffts/126036/6 "2025-02-18T18:34:00Z")

</div>

If I recall correctly, there’s no real2real test transforms in AbstractFFTs.jl. So we’d have to make like a DCT to test the remaining lines. This can probably be done via the test FFT on a double length vector but I couldn’t remember the formula when I stopped working on it

---

<div class="post-metadata">

**Author:** ![cgeoga](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cgeoga/32/216186_2.png) [@cgeoga](https://discourse.julialang.org/u/cgeoga)\
**Post date:** [February 18, 2025, 6:36pm UTC](https://discourse.julialang.org/t/forward-automatic-differentiation-ffts/126036/7 "2025-02-18T18:36:32Z")

</div>

~~If you need a solution right now, don’t want to write rules yourself, and can give up some speed, you could use [`GenericFFTs.jl`](https://github.com/JuliaApproximation/GenericFFT.jl). I haven’t used the package in years, but if memory serves that is also an option that fits your needs.~~

EDIT: no, I just tried and it does have some type restrictions and ForwardDiff doesn’t work out of the box. My apologies.

---

<div class="post-metadata">

**Author:** ![marcobonici](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcobonici/32/20549_2.png) [@marcobonici](https://discourse.julialang.org/u/marcobonici)\
**Post date:** [February 18, 2025, 8:18pm UTC](https://discourse.julialang.org/t/forward-automatic-differentiation-ffts/126036/8 "2025-02-18T20:18:27Z")

</div>

Ok, so basically you are saying it has already been implemented but there is lack of proper testing one specific case.  
I’ll look into it.

---

<div class="post-metadata">

**Author:** ![marcobonici](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcobonici/32/20549_2.png) [@marcobonici](https://discourse.julialang.org/u/marcobonici)\
**Post date:** [February 18, 2025, 10:41pm UTC](https://discourse.julialang.org/t/forward-automatic-differentiation-ffts/126036/9 "2025-02-18T22:41:11Z")

</div>

Ok,  
I quickly wrote a stupid piece of code, trying the type FFTs I need (DHT and REDFT00).  
ForwardDiff.jl is working when I do NOT use a plan, with a plan it is failing (although I would be completely satisfied even without the plan). A quick check with FiniteDifferences shows a great agreement.  
I’ll now go through

> <https://github.com/JuliaMath/AbstractFFTs.jl/pull/141>
>
> I need an example real-to-real transform in https://github.com/JuliaMath/Abstrac…tFFTs.jl/pull/138
> 
> I'll propose it in this PR since it can be merged separately.

> <https://github.com/JuliaMath/AbstractFFTs.jl/pull/138>
>
> This is moving type-piracy code from https://github.com/JuliaApproximation/FastT…ransformsForwardDiff.jl to an extension here.

So, my understading is that we need to fix 141, to then use the developed tests into 138.  
It will take a bit, but I will get back with some proposed fixes.  
Thx again!

---

<div class="post-metadata">

**Author:** ![dlfivefifty](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlfivefifty/32/1959_2.png) [@dlfivefifty](https://discourse.julialang.org/u/dlfivefifty)\
**Post date:** [February 19, 2025, 10:43am UTC](https://discourse.julialang.org/t/forward-automatic-differentiation-ffts/126036/10 "2025-02-19T10:43:28Z")

</div>

Btw a simpler solution might be to move these extensions to ForwardDiff.jl, as then we can use FFTW.jl to test code in the AbstractFFTs.jl extension.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [February 19, 2025, 11:04am UTC](https://discourse.julialang.org/t/forward-automatic-differentiation-ffts/126036/11 "2025-02-19T11:04:28Z")

</div>

There’s no universal truth as to where package extensions belong, but the answer is usually in the less universal or less widely-used package. Which here means I’d probably aim to put ForwardDiff extensions in the FFT packages, not the other way around.

---

<div class="post-metadata">

**Author:** ![dlfivefifty](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlfivefifty/32/1959_2.png) [@dlfivefifty](https://discourse.julialang.org/u/dlfivefifty)\
**Post date:** [February 19, 2025, 11:15am UTC](https://discourse.julialang.org/t/forward-automatic-differentiation-ffts/126036/12 "2025-02-19T11:15:52Z")

</div>

Saying that automatic differentiation is more used than FFTs would shock 99% of all of scientific computing / Matlab users 😂

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [February 19, 2025, 11:25am UTC](https://discourse.julialang.org/t/forward-automatic-differentiation-ffts/126036/13 "2025-02-19T11:25:29Z")

</div>

Indeed, but speaking only about Julia packages, saying that ForwardDiff.jl is used more widely than FFTW.jl is supported (albeit barely) by [juliahub.com](http://juliahub.com) statistics 🙂

---

<div class="post-metadata">

**Author:** ![dlfivefifty](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlfivefifty/32/1959_2.png) [@dlfivefifty](https://discourse.julialang.org/u/dlfivefifty)\
**Post date:** [February 22, 2025, 10:18pm UTC](https://discourse.julialang.org/t/forward-automatic-differentiation-ffts/126036/14 "2025-02-22T22:18:37Z")

</div>

I know. But when I tell other numerical analysts I teach dual numbers/ automatic differentiation they give me grief because it’s “niche”