# FFT is too fast compared to assignment in CUDA

**URL:** <https://discourse.julialang.org/t/fft-is-too-fast-compared-to-assignment-in-cuda/70353>\
**Category:** GPU\
**Tags:** fftw, cuda\
**Created:** [October 25, 2021, 8:14pm UTC](https://discourse.julialang.org/t/fft-is-too-fast-compared-to-assignment-in-cuda/70353 "2021-10-25T20:14:59Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![wsshin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wsshin/32/360_2.png) [@wsshin](https://discourse.julialang.org/u/wsshin)\
**Post date:** [October 25, 2021, 8:15pm UTC](https://discourse.julialang.org/t/fft-is-too-fast-compared-to-assignment-in-cuda/70353/1 "2021-10-25T20:15:00Z")

</div>

FFT is a pretty fast algorithm, but its performance on CUDA seems even comparable to simple element-wise assignment. I am wondering if this is something expected.

Here is the contents of a performance test code named `test_fft_vs_assign.jl` for a fairly large number of sampling points (`N = 2^20`):

```julia
using CUDA
using FFTW
using BenchmarkTools

n = 20
N = 2^n

x = rand(ComplexF64, N)
y = similar(x)
F = plan_fft!(x)

cx = similar(cu(x), ComplexF64) # make sure eltype(cx) = ComplexF64
copyto!(cx, x) # copy contents of x to cx
cy = similar(cx)
cF = plan_fft!(cx)

print("CPU assignment:")
@btime $y .= $x

print("CPU assignment + in-place planned FFT:")
@btime ($y .= $x; $F * $y) # assignment first to keep input same

print("GPU assignment:")
@btime $cy .= $cx

print("GPU assignment + in-place planned FFT:")
@btime ($cy .= $cx; $cF * $cy)

```

and here is the result:

```julia-repl
julia> VERSION
v"1.6.2-pre.0"

(@v1.6) pkg> st CUDA
      Status `~/.julia/environments/v1.6/Project.toml`
  [052768ef] CUDA v3.4.2

(@v1.6) pkg> st FFTW
      Status `~/.julia/environments/v1.6/Project.toml`
  [7a1cc6ca] FFTW v1.4.5

julia> include("test_fft_vs_assign.jl")
CPU assignment: 2.442 ms (0 allocations: 0 bytes)
CPU assignment + in-place planned FFT: 16.181 ms (0 allocations: 0 bytes) # FFT alone: 16.181 − 2.442 = 13.739 ms
GPU assignment: 3.903 μs (27 allocations: 1.47 KiB)
GPU assignment + in-place planned FFT: 8.951 μs (28 allocations: 1.50 KiB) # FFT alone: 8.951 − 3.903 = 5.048 µs

```

So, in this specific experiment,

- on CPU: FFT of a vector is slower than element-wise assignment by a factor of 13.739 ms / 2.442 ms ≈ 5.6.
- on GPU: FFT of a vector is slower than element-wise assignment by a factor of 5.048 µs / 3.903 µs ≈ 1.3.

This means that FFT is nearly as cheap as element-wise assignment on GPU.

I am implementing an algorithm in which FFT operations are known to be the most time-consuming part. So, I have been trying to minimize the number of FFTs in my implementation of the algorithm. But the algorithm uses lots of basic element-wise operations (assignment, addition, multiplication, …) as well between CUDA vectors. Because those basic operations are nearly as expensive as FFT, it seems that I don’t have to be too concerned about reducing the number of FFTs in implementing the algorithm on GPU.

Of course, the result would be different for different values of `N`, but I am wondering if the above conclusion is more or less correct in dealing with FFT on GPU.

---

<div class="post-metadata">

**Author:** ![maedoc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maedoc/32/8490_2.png) [@maedoc](https://discourse.julialang.org/u/maedoc)\
**Post date:** [October 25, 2021, 9:10pm UTC](https://discourse.julialang.org/t/fft-is-too-fast-compared-to-assignment-in-cuda/70353/2 "2021-10-25T21:10:36Z")

</div>

1M element FFT is not much for a modern GPU. [How bad is it to launch many small kernels in CUDA? - Stack Overflow](https://stackoverflow.com/a/27038751) suggests you might just be seeing a kernel launch overhead floor. You might try larger problem sizes for better comparison. I’d also suggest computing expected copy performance based on your device’s global memory bandwidth.

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [October 27, 2021, 4:31pm UTC](https://discourse.julialang.org/t/fft-is-too-fast-compared-to-assignment-in-cuda/70353/3 "2021-10-27T16:31:38Z")

</div>

Try again with synchronization on the CUDA side to make sure you’re capturing the full execution time: [Profiling · CUDA.jl](https://cuda.juliagpu.org/stable/development/profiling/#Time-measurements). It’s possible only the async launch time is being measured as @maedoc mentioned.

---

<div class="post-metadata">

**Author:** ![maedoc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maedoc/32/8490_2.png) [@maedoc](https://discourse.julialang.org/u/maedoc)\
**Post date:** [October 28, 2021, 8:23pm UTC](https://discourse.julialang.org/t/fft-is-too-fast-compared-to-assignment-in-cuda/70353/4 "2021-10-28T20:23:29Z")

</div>

Thanks for the tip for `CUDA.@sync`. On my card, with the sync, the memcpy takes about 100 us, while the FFT takes 924 us, for 1M elements. This makes more sense.