# Is CUDA.jl and FFTW threadsafe?
**URL:** https://discourse.julialang.org/t/is-cuda-jl-and-fftw-threadsafe/99219
**Category:** GPU
**Tags:** fftw, multithreading, cuda, fft
**Created:** [May 22, 2023, 1:20pm UTC](https://discourse.julialang.org/t/is-cuda-jl-and-fftw-threadsafe/99219 "2023-05-22T13:20:46Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@jpdoane](https://discourse.julialang.org/u/jpdoane)
#### Post date: [May 22, 2023, 1:20pm UTC](https://discourse.julialang.org/t/is-cuda-jl-and-fftw-threadsafe/99219/1 "2023-05-22T13:20:46Z")
I am getting the following error when using CUDA.jl FFTW plans in multiple threads. Is this interface not threadsafe? If not, do I just need a mutex around plan\_fft!(), or might the actual fft be not threadsafe as well?
```julia
using CUDA, FFTW
function gpu_fft_thread()
try
X = CUDA.randn(ComplexF32, 1024,1024)
myfft = plan_fft!(X, 1)
myfft * X
catch e
error(e)
end
return nothing
end
function run_fft_threads()
try
for nn=1:10
tids = [Threads.@spawn gpu_fft_thread() for nn=1:10]
while !all(istaskdone.(tids))
yield();
end
end
catch e
error(e)
end
end
run_fft_threads()
```
Output:
```julia
error in running finalizer: ErrorException("val already in a list")
error at ./error.jl:35
push! at ./linked_list.jl:53 [inlined]
_wait2 at ./condition.jl:87
#wait#621 at ./condition.jl:127
wait at ./condition.jl:125 [inlined]
slowlock at ./lock.jl:156
lock at ./lock.jl:147 [inlined]
lock at ./lock.jl:227
push! at /home/jon/.julia/packages/CUDA/p5OVK/lib/utils/cache.jl:72 [inlined]
cufftReleasePlan at /home/jon/.julia/packages/CUDA/p5OVK/lib/cufft/wrappers.jl:158 [inlined]
#137 at /home/jon/.julia/packages/CUDA/p5OVK/lib/cufft/fft.jl:30 [inlined]
#context!#59 at /home/jon/.julia/packages/CUDA/p5OVK/lib/cudadrv/state.jl:170 [inlined]
context! at /home/jon/.julia/packages/CUDA/p5OVK/lib/cudadrv/state.jl:165 [inlined]
unsafe_free! at /home/jon/.julia/packages/CUDA/p5OVK/lib/cufft/fft.jl:29
unknown function (ip: 0x7fefe0126a62)
_jl_invoke at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2940
run_finalizer at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gc.c:417
jl_gc_run_finalizers_in_list at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gc.c:507
run_finalizers at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gc.c:553
enable_finalizers at ./gcutils.jl:126 [inlined]
unlock at ./locks-mt.jl:68 [inlined]
push! at ./task.jl:703
enq_work at ./task.jl:783
yield at ./task.jl:862
run_fft_threads at /home/jon/processing_analysis/fftw_threads.jl:19
unknown function (ip: 0x7fefe015402f)
_jl_invoke at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2940
jl_apply at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/julia.h:1879 [inlined]
do_call at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:126
eval_value at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:226
eval_stmt_value at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:177 [inlined]
eval_body at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:624
jl_interpret_toplevel_thunk at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:762
top-level scope at /home/jon/processing_analysis/fftw_threads.jl:27
jl_toplevel_eval_flex at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/toplevel.c:912
jl_toplevel_eval_flex at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/toplevel.c:856
ijl_toplevel_eval_in at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/toplevel.c:971
eval at ./boot.jl:370 [inlined]
include_string at ./loading.jl:1864
_jl_invoke at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2940
_include at ./loading.jl:1924
include at ./client.jl:478
unknown function (ip: 0x7fefe00998a2)
_jl_invoke at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2940
jl_apply at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/julia.h:1879 [inlined]
do_call at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:126
eval_value at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:226
eval_stmt_value at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:177 [inlined]
eval_body at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:624
jl_interpret_toplevel_thunk at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:762
top-level scope at REPL[1]:1
jl_toplevel_eval_flex at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/toplevel.c:912
jl_toplevel_eval_flex at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/toplevel.c:856
jl_toplevel_eval_flex at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/toplevel.c:856
ijl_toplevel_eval_in at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/toplevel.c:971
eval at ./boot.jl:370 [inlined]
eval at ./Base.jl:68 [inlined]
repleval at /home/jon/.vscode-server/extensions/julialang.language-julia-1.47.2/scripts/packages/VSCodeServer/src/repl.jl:222
#107 at /home/jon/.vscode-server/extensions/julialang.language-julia-1.47.2/scripts/packages/VSCodeServer/src/repl.jl:186
unknown function (ip: 0x7fefe009982f)
_jl_invoke at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2940
with_logstate at ./logging.jl:514
with_logger at ./logging.jl:626 [inlined]
#106 at /home/jon/.vscode-server/extensions/julialang.language-julia-1.47.2/scripts/packages/VSCodeServer/src/repl.jl:187
unknown function (ip: 0x7fefe009960f)
_jl_invoke at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2940
jl_apply at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/julia.h:1879 [inlined]
jl_f__call_latest at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/builtins.c:774
#invokelatest#2 at ./essentials.jl:816 [inlined]
invokelatest at ./essentials.jl:813
unknown function (ip: 0x7fefe0098772)
_jl_invoke at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2940
jl_apply at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/julia.h:1879 [inlined]
do_apply at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/builtins.c:730
macro expansion at /home/jon/.vscode-server/extensions/julialang.language-julia-1.47.2/scripts/packages/VSCodeServer/src/eval.jl:34 [inlined]
#61 at ./task.jl:514
unknown function (ip: 0x7fefe007bb9f)
_jl_invoke at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2940
jl_apply at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/julia.h:1879 [inlined]
start_task at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/task.c:1092
```
The error is not fully deterministic, but occurs most of the time on my machine with the above code. If you have difficultly reproducing, you may try running multiple times or increase the number of loops/threads.
---
### Author:  [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)
#### Post date: [May 22, 2023, 1:37pm UTC](https://discourse.julialang.org/t/is-cuda-jl-and-fftw-threadsafe/99219/2 "2023-05-22T13:37:51Z")
Hi,
can confirm the crash.
just to clarify, you don’t need to load FFTW.jl but instead [CUDA.CUFFT](https://github.com/JuliaGPU/CUDA.jl/tree/master/lib/cufft). is enough.
FFTW.jl only handles `Array`s whereas CUDA.CUFFT handles `CuArray`s.
I’m wondering, why don’t you use batched FFTs. This ensures that the FFT is properly parallelized on the GPU which would not happen with a for-loop. Or doesn’t this work in your application?
```julia
julia> using CUDA, CUDA.CUFFT
julia> arr = CUDA.randn(32, 124, 124);
julia> p = plan_fft(arr, (1,2))
CUFFT complex forward plan for 32×124×124 CuArray of ComplexF32
julia> CUDA.@time CUDA.@sync p * arr; # FFT along first and second dim
0.026062 seconds (13.54 k CPU allocations: 958.493 KiB) (3 GPU allocations: 11.262 MiB, 0.07% memmgmt time)
julia> CUDA.@time CUDA.@sync p * arr; # FFT along first and second dim
0.000897 seconds (92 CPU allocations: 4.406 KiB) (3 GPU allocations: 11.262 MiB, 35.61% memmgmt time)
```
---
### Author:  [@jpdoane](https://discourse.julialang.org/u/jpdoane)
#### Post date: [May 22, 2023, 2:05pm UTC](https://discourse.julialang.org/t/is-cuda-jl-and-fftw-threadsafe/99219/3 "2023-05-22T14:05:22Z")
The code snippet is a simple MWE just designed to reproduce the crash. My actual problem is more complicated and organized a bit differently – I am doing more than just ffts and am using threads to maintain separate GPU streams as well as parallelization of CPU bound tasks.
Thanks for the CUFFT hint, I had thought I needed to load FFTW to access the plan interface. I do seem to get the same crash with CUDA.CUFFT though.
---
### Author:  [@maleadt](https://discourse.julialang.org/u/maleadt)
#### Post date: [May 22, 2023, 2:17pm UTC](https://discourse.julialang.org/t/is-cuda-jl-and-fftw-threadsafe/99219/4 "2023-05-22T14:17:32Z")
The functionality in question was supposed to be thread-safe – all accesses to `cache.active_handles` are behind a lock – and it’s the `lock` operation that’s failing here…
```julia
error in running finalizer: ErrorException("val already in a list")
error at ./error.jl:35
push! at ./linked_list.jl:53 [inlined]
_wait2 at ./condition.jl:87
#wait#621 at ./condition.jl:127
wait at ./condition.jl:125 [inlined]
slowlock at ./lock.jl:156
lock at ./lock.jl:147 [inlined]
lock at ./lock.jl:227
push! at /home/jon/.julia/packages/CUDA/p5OVK/lib/utils/cache.jl:72 [inlined]
```
Please open an issue, and be sure to mention which Julia version you’re using.
---
### Author:  [@jpdoane](https://discourse.julialang.org/u/jpdoane)
#### Post date: [May 22, 2023, 2:44pm UTC](https://discourse.julialang.org/t/is-cuda-jl-and-fftw-threadsafe/99219/5 "2023-05-22T14:44:21Z")
Thanks for the quick response. I submitted an issue
>
>
> The CUDA.CUFFT.plan\_fft!() appears to have a thread locking issue:
>
> \`\`\`julia
> …using CUDA, CUDA.CUFFT
>
> function gpu\_fft\_thread()
> try
> X = CUDA.randn(ComplexF32, 1024,1024)
> myfft = plan\_fft!(X, 1)
> myfft \* X
> catch e
> error(e)
> end
> return nothing
> end
>
> function run\_fft\_threads()
> try
> for nn=1:10
> tids = \[Threads.@spawn gpu\_fft\_thread() for nn=1:10\]
> while !all(istaskdone.(tids))
> yield();
> end
> end
> catch e
> error(e)
> end
> end
>
> run\_fft\_threads()
> \`\`\`
> generates:
> \`\`\`
> error in running finalizer: ErrorException("val already in a list")
> error at ./error.jl:35
> push! at ./linked\_list.jl:53 \[inlined\]
> \_wait2 at ./condition.jl:87
> \#wait#621 at ./condition.jl:127
> wait at ./condition.jl:125 \[inlined\]
> slowlock at ./lock.jl:156
> lock at ./lock.jl:147 \[inlined\]
> lock at ./lock.jl:227
> push! at /home/jon/.julia/packages/CUDA/p5OVK/lib/utils/cache.jl:72 \[inlined\]
> cufftReleasePlan at /home/jon/.julia/packages/CUDA/p5OVK/lib/cufft/wrappers.jl:158 \[inlined\]
> \#137 at /home/jon/.julia/packages/CUDA/p5OVK/lib/cufft/fft.jl:30 \[inlined\]
> \#context!#59 at /home/jon/.julia/packages/CUDA/p5OVK/lib/cudadrv/state.jl:170 \[inlined\]
> context! at /home/jon/.julia/packages/CUDA/p5OVK/lib/cudadrv/state.jl:165 \[inlined\]
> unsafe\_free! at /home/jon/.julia/packages/CUDA/p5OVK/lib/cufft/fft.jl:29
> unknown function (ip: 0x7fefe0126a62)
> \_jl\_invoke at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2758 \[inlined\]
> ijl\_apply\_generic at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2940
> run\_finalizer at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gc.c:417
> jl\_gc\_run\_finalizers\_in\_list at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gc.c:507
> run\_finalizers at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gc.c:553
> enable\_finalizers at ./gcutils.jl:126 \[inlined\]
> unlock at ./locks-mt.jl:68 \[inlined\]
> push! at ./task.jl:703
> enq\_work at ./task.jl:783
> yield at ./task.jl:862
> run\_fft\_threads at /home/jon/processing\_analysis/fftw\_threads.jl:19
> unknown function (ip: 0x7fefe015402f)
> \_jl\_invoke at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2758 \[inlined\]
> ijl\_apply\_generic at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2940
> jl\_apply at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/julia.h:1879 \[inlined\]
> do\_call at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:126
> eval\_value at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:226
> eval\_stmt\_value at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:177 \[inlined\]
> eval\_body at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:624
> jl\_interpret\_toplevel\_thunk at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:762
> top-level scope at /home/jon/processing\_analysis/fftw\_threads.jl:27
> jl\_toplevel\_eval\_flex at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/toplevel.c:912
> jl\_toplevel\_eval\_flex at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/toplevel.c:856
> ijl\_toplevel\_eval\_in at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/toplevel.c:971
> eval at ./boot.jl:370 \[inlined\]
> include\_string at ./loading.jl:1864
> \_jl\_invoke at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2758 \[inlined\]
> ijl\_apply\_generic at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2940
> \_include at ./loading.jl:1924
> include at ./client.jl:478
> unknown function (ip: 0x7fefe00998a2)
> \_jl\_invoke at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2758 \[inlined\]
> ijl\_apply\_generic at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2940
> jl\_apply at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/julia.h:1879 \[inlined\]
> do\_call at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:126
> eval\_value at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:226
> eval\_stmt\_value at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:177 \[inlined\]
> eval\_body at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:624
> jl\_interpret\_toplevel\_thunk at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/interpreter.c:762
> top-level scope at REPL\[1\]:1
> jl\_toplevel\_eval\_flex at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/toplevel.c:912
> jl\_toplevel\_eval\_flex at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/toplevel.c:856
> jl\_toplevel\_eval\_flex at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/toplevel.c:856
> ijl\_toplevel\_eval\_in at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/toplevel.c:971
> eval at ./boot.jl:370 \[inlined\]
> eval at ./Base.jl:68 \[inlined\]
> repleval at /home/jon/.vscode-server/extensions/julialang.language-julia-1.47.2/scripts/packages/VSCodeServer/src/repl.jl:222
> \#107 at /home/jon/.vscode-server/extensions/julialang.language-julia-1.47.2/scripts/packages/VSCodeServer/src/repl.jl:186
> unknown function (ip: 0x7fefe009982f)
> \_jl\_invoke at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2758 \[inlined\]
> ijl\_apply\_generic at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2940
> with\_logstate at ./logging.jl:514
> with\_logger at ./logging.jl:626 \[inlined\]
> \#106 at /home/jon/.vscode-server/extensions/julialang.language-julia-1.47.2/scripts/packages/VSCodeServer/src/repl.jl:187
> unknown function (ip: 0x7fefe009960f)
> \_jl\_invoke at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2758 \[inlined\]
> ijl\_apply\_generic at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2940
> jl\_apply at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/julia.h:1879 \[inlined\]
> jl\_f\_\_call\_latest at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/builtins.c:774
> \#invokelatest#2 at ./essentials.jl:816 \[inlined\]
> invokelatest at ./essentials.jl:813
> unknown function (ip: 0x7fefe0098772)
> \_jl\_invoke at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2758 \[inlined\]
> ijl\_apply\_generic at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2940
> jl\_apply at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/julia.h:1879 \[inlined\]
> do\_apply at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/builtins.c:730
> macro expansion at /home/jon/.vscode-server/extensions/julialang.language-julia-1.47.2/scripts/packages/VSCodeServer/src/eval.jl:34 \[inlined\]
> \#61 at ./task.jl:514
> unknown function (ip: 0x7fefe007bb9f)
> \_jl\_invoke at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2758 \[inlined\]
> ijl\_apply\_generic at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/gf.c:2940
> jl\_apply at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/julia.h:1879 \[inlined\]
> start\_task at /cache/build/default-amdci4-4/julialang/julia-release-1-dot-9/src/task.c:1092
> \`\`\`
>
> Discourse thread:
> https://discourse.julialang.org/t/is-cuda-jl-and-fftw-threadsafe/99219/4?u=jpdoane
>
> version info:
> \`\`\`
> CUDA runtime 12.1, artifact installation
> CUDA driver 12.1
> NVIDIA driver 525.78.1, originally for CUDA 12.0
>
> Libraries:
> \- CUBLAS: 12.1.3
> \- CURAND: 10.3.2
> \- CUFFT: 11.0.2
> \- CUSOLVER: 11.4.5
> \- CUSPARSE: 12.1.0
> \- CUPTI: 18.0.0
> \- NVML: 12.0.0+525.78.1
>
> Toolchain:
> \- Julia: 1.9.0
> \- LLVM: 14.0.6
> \- PTX ISA support: 3.2, 4.0, 4.1, 4.2, 4.3, 5.0, 6.0, 6.1, 6.3, 6.4, 6.5, 7.0, 7.1, 7.2, 7.3, 7.4, 7.5
> \- Device capability support: sm\_37, sm\_50, sm\_52, sm\_53, sm\_60, sm\_61, sm\_62, sm\_70, sm\_72, sm\_75, sm\_80, sm\_86
> \`\`\`