# ANN: InplaceRealFFT.jl : inplace real-to-complex and complex-to-real FFT

**URL:** <https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116>\
**Category:** Numerics\
**Tags:** fftw\
**Created:** [January 2, 2018, 9:30pm UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116 "2018-01-02T21:30:43Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![favba](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/favba/32/2735_2.png) [@favba](https://discourse.julialang.org/u/favba)\
**Post date:** [January 2, 2018, 9:30pm UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/1 "2018-01-02T21:30:43Z")

</div>

Hello everyone!

I’ve created the [InplaceRealFFT.jl](https://github.com/favba/InplaceRealFFT.jl) module to expose FFTW’s inplace real-to-complex (`rfft!`) and complex-to-real (`irfft!`) transformations. Currently, Julia only provides inplace version of the transform for complex-to-complex transformations (`fft!` and `ifft!`). I’ve been using this module for a while but it was only now that I’ve polished it to be used by others.

It is not yet registered, so in order to install it one must `Pkg.clone("https://github.com/favba/InplaceRealFFT.jl")`.  
It is still not well documented, but it works almost the same as the out-of-place transform `rfft`.

I believe this package can be useful to anyone performing FFTs of real data.

* * *

I’d love to have some feedback from the community, If people find this useful I could register it. Also, if people think this package is worth living in [FFTW.jl](https://github.com/JuliaMath/FFTW.jl) as a submodule, I’d happily make a PR.

---

<div class="post-metadata">

**Author:** ![dlfivefifty](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlfivefifty/32/1959_2.png) [@dlfivefifty](https://discourse.julialang.org/u/dlfivefifty)\
**Post date:** [January 2, 2018, 9:47pm UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/2 "2018-01-02T21:47:26Z")

</div>

+1 for making a pull request in to FFTW.jl

---

<div class="post-metadata">

**Author:** ![Paulo\_Jabardo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paulo_jabardo/32/3196_2.png) [@Paulo\_Jabardo](https://discourse.julialang.org/u/Paulo_Jabardo)\
**Post date:** [January 2, 2018, 9:59pm UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/3 "2018-01-02T21:59:30Z")

</div>

Definitely a PR to FFTW.jl!

Nice work.

---

<div class="post-metadata">

**Author:** ![favba](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/favba/32/2735_2.png) [@favba](https://discourse.julialang.org/u/favba)\
**Post date:** [January 3, 2018, 5:46pm UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/4 "2018-01-03T17:46:53Z")

</div>

I opened an [issue](https://github.com/JuliaMath/FFTW.jl/issues/52) for FFTW.jl to see what folks think of providing this natively.

---

<div class="post-metadata">

**Author:** ![favba](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/favba/32/2735_2.png) [@favba](https://discourse.julialang.org/u/favba)\
**Post date:** [January 3, 2018, 6:07pm UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/5 "2018-01-03T18:07:48Z")

</div>

I’d also like to ask something to someone more knowledgeable on the matter.

In order to make the module work on Julia v0.7 I had to stop using `reinterpret` and started using [this trick](https://discourse.julialang.org/t/big-overhead-with-the-new-lazy-reshape-reinterpret/7635/6).  
[https://github.com/favba/InplaceRealFFT.jl/blob/2d6a82fc0b412bb7a072a930af4a07b63e8e9105/src/InplaceRealFFT.jl#L29](https://github.com/favba/InplaceRealFFT.jl/blob/2d6a82fc0b412bb7a072a930af4a07b63e8e9105/src/InplaceRealFFT.jl#L29)

Could that lead to problems? I though that maybe, because both views of the data are living in the same struct, that could make it safer to use.

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [January 3, 2018, 6:50pm UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/6 "2018-01-03T18:50:54Z")

</div>

I think you should not run into any gc problems with this, and this looks like the most possible safe usecase.

If a user decides to keep the real view and let the PaddedArray die, then he can corrupt memory; this is ok (just don’t do this!).

If a user works on both the real and complex views at the same time there might be aliasing issues, but your code does not do such things (and these hypothetical aliasing issues are probably only my paranoia).

Caveat: My understanding of the internals is limited. Maybe others know better.

Option: Make the real array the owner of the memory. Then it is safe to take out the real part and run away with it, and unsafe to take out the complex view and run away.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [January 3, 2018, 7:01pm UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/7 "2018-01-03T19:01:15Z")

</div>

No you must not do it. There’s absolutely no guarantee that the data will be valid

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [January 3, 2018, 7:32pm UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/8 "2018-01-03T19:32:21Z")

</div>

Would you mind elaborating what the problem is?

From you answer, I’d guess you think that a use-after-free might happen in some cases; in what kind of cases?

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [January 3, 2018, 7:47pm UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/9 "2018-01-03T19:47:24Z")

</div>

> [@foobar\_lv2](#):
>
> If a user decides to keep the real view and let the PaddedArray die

Compiler can do this, easily.

> [@foobar\_lv2](#):
>
> in what kind of cases?

In general, just don’t do this kind of analysis. All what you can ever do is to assume certain compiler limitation and rely on implementation details that can easily change.

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [January 3, 2018, 8:37pm UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/10 "2018-01-03T20:37:16Z")

</div>

> [@yuyichao](#):
>
> In general, just don’t do this kind of analysis. All what you can ever do is to assume certain compiler limitation and rely on implementation details that can easily change.

That’s a reasonable standard for security-relevant code and for base / quasi-stdlib code.

I have a feeling that performance now is more important than possible breakage in future updates for much numerics code (especially since I would expect reinterpreted arrays to become fast soon), but it’s not my call to make.

Nevertheless, thanks!

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [January 3, 2018, 9:06pm UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/11 "2018-01-03T21:06:38Z")

</div>

No it’s not breakage in future, it’s breakage when the code is used in a way that the compiler can already optimize.

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [January 3, 2018, 9:26pm UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/12 "2018-01-03T21:26:35Z")

</div>

So the proper guard would be to @noinline all scary functions, and then just read the @code\_llvm or test to see whether everything is fine?

I mean, the overhead of a couple function calls is negligible compared to fft computation.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [January 3, 2018, 9:30pm UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/13 "2018-01-03T21:30:03Z")

</div>

No, the propery way is to use `@gc_preserve` in this function and every single users.

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [January 3, 2018, 9:36pm UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/14 "2018-01-03T21:36:31Z")

</div>

Thanks a lot!

Now I feel stupid. Since @favba has a quite small lib, this is actually feasible (there are only 3-4 dangerous functions to annotate).

---

<div class="post-metadata">

**Author:** ![favba](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/favba/32/2735_2.png) [@favba](https://discourse.julialang.org/u/favba)\
**Post date:** [January 3, 2018, 10:07pm UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/15 "2018-01-03T22:07:43Z")

</div>

Thank you guys, for the help.

> [@foobar\_lv2](#):
>
> Option: Make the real array the owner of the memory. Then it is safe to take out the real part and run away with it, and unsafe to take out the complex view and run away.

It does make a lot more sense to use the real view as the “parent” of the others. I believe most of the use for `rfft!` starts with real data, perform some calculation in Fourier space, then ends again in real space.

> [@yuyichao](#):
>
> No you must not do it. There’s absolutely no guarantee that the data will be valid

Since we are trying to push this functionality to FFTW.jl , I’ll try to stick to the supported `ReinterpretedArray`, which will hopefully became faster in the future. For some reason it wasn’t working but I really didn’t dove deep to find the problem as I found the trick to get the same past behavior.

---

<div class="post-metadata">

**Author:** ![favba](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/favba/32/2735_2.png) [@favba](https://discourse.julialang.org/u/favba)\
**Post date:** [January 4, 2018, 12:20am UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/16 "2018-01-04T00:20:33Z")

</div>

I’m changing my mind again. In order to use the new `ReinterpretArray` I’d have to change FFTW.jl internals, which I got away with so far and might be out of my league. Also, right now I’m getting a 200x slowdown on a `sum` with the `ReinterpretArray`.

> [@yuyichao](#):
>
> No, the propery way is to use @gc\_preserve in this function and every single users.

Sorry to bother again, my understanding of this subject is very limited. Do you mean here that every time the unsafe\_wraped array is used, not only in the module itself, but also in any code used by a prospective user, there must be a `@gc_preserve`? In other words, it would be impossible to make usage of the module by third parties inherently safe?

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [January 4, 2018, 1:22am UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/17 "2018-01-04T01:22:26Z")

</div>

> [@favba](#):
>
> Do you mean here that every time the unsafe\_wraped array is used, not only in the module itself, but also in any code used by a prospective user, there must be a @gc\_preserve?

Correct.

> [@favba](#):
>
> In other words, it would be impossible to make usage of the module by third parties inherently safe?

If that array is to be used directly by user code, then yes, that is the case. It’s ok if the user will never access that array directly.

---

<div class="post-metadata">

**Author:** ![favba](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/favba/32/2735_2.png) [@favba](https://discourse.julialang.org/u/favba)\
**Post date:** [January 4, 2018, 2:51am UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/18 "2018-01-04T02:51:08Z")

</div>

Finally, to see if I really understand it: Then, this risky behavior of the trick was already present in the `reinterpret` method provided in Julia v0.6 and this is the reason (or at least one of them) for the new `ReinterpretArray` type, right?

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [January 4, 2018, 2:57am UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/19 "2018-01-04T02:57:12Z")

</div>

No.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [January 4, 2018, 3:04am UTC](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116/20 "2018-01-04T03:04:15Z")

</div>

I think the reason for the new type is so that Julia can assume that arrays with different element types never alias.

[Next page](https://discourse.julialang.org/t/ann-inplacerealfft-jl-inplace-real-to-complex-and-complex-to-real-fft/8116.md?page=2)