# GPU is slower than CPU for findall on a CuArray

**URL:** <https://discourse.julialang.org/t/gpu-is-slower-than-cpu-for-findall-on-a-cuarray/128916>\
**Category:** GPU\
**Created:** [May 12, 2025, 4:23am UTC](https://discourse.julialang.org/t/gpu-is-slower-than-cpu-for-findall-on-a-cuarray/128916 "2025-05-12T04:23:35Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jtoledom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jtoledom/32/216799_2.png) [@jtoledom](https://discourse.julialang.org/u/jtoledom)\
**Post date:** [May 12, 2025, 4:23am UTC](https://discourse.julialang.org/t/gpu-is-slower-than-cpu-for-findall-on-a-cuarray/128916/1 "2025-05-12T04:23:35Z")

</div>

I’m writing a code to generate Von Mises random numbers based on this method which is free of accept/reject.  
It is based on this [paper](https://ieeexplore.ieee.org/document/4568457).

This issue appears to be with `findall`. Anyone can reproduce the behaviour by using this simple [script](https://github.com/jquetzalcoatl/RKM/blob/main/notebook/random_numbers_with_vonMises.jl).

To generate 500k random numbers Von Mises distributed, line 11 runs the whole algorithm and it take a several seconds (~40s). However, if I break that same function into two separate functions (lines 13 and 14 ), function \_A runs very fast (0.06s) and function \_B takes about 40s.

In particular, in function \_B, line 198 which is `index1 = CUDA.findall(r .<= p[1,:])` is what’s taking the chunk of time.

I also noticed that if I run line 14 (without running line 13), it takes about 0.02s. Which makes me think that perhaps when the CuArray `p` is generated in line 7, it is somehow cached, whereas when it is generate by function \_A in line 13, it is not cached and that’s what creates this huge lag.

Can anyone point me in the right direction?

Thanks!

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [May 12, 2025, 8:15am UTC](https://discourse.julialang.org/t/gpu-is-slower-than-cpu-for-findall-on-a-cuarray/128916/2 "2025-05-12T08:15:42Z")

</div>

GPU operations are asynchronous wrt. the CPU, so some may appear to take little time while others “soak up” all of the time spent in previous GPU operations. You should either use time operations individually by surrounding them using `CUDA.@sync` (forcing synchronization), or simply run your application under a GPU-aware profiler (e.g. `CUDA.@profile`) which will correctly report device-time spent. To add information about your CPU-side operations (e.g. the call to `findall`) to the trace, consider using NVTX.jl’s `@range`.

---

<div class="post-metadata">

**Author:** ![jtoledom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jtoledom/32/216799_2.png) [@jtoledom](https://discourse.julialang.org/u/jtoledom)\
**Post date:** [May 14, 2025, 12:02am UTC](https://discourse.julialang.org/t/gpu-is-slower-than-cpu-for-findall-on-a-cuarray/128916/3 "2025-05-14T00:02:17Z")

</div>

Using CUDA@sync and CUDA.@time I was able to pinpoint correctly the step in my algorithm that was chewing all the algorithm time. It wasn’t `findall`. After fixing it, the code generates Von Mises random numbers with random parameters on the order of 100ns.

Thnx @maleadt !