# Sum is very slow (and I can't figure out why)

**URL:** <https://discourse.julialang.org/t/sum-is-very-slow-and-i-cant-figure-out-why/52791>\
**Category:** GPU\
**Created:** [January 3, 2021, 7:15pm UTC](https://discourse.julialang.org/t/sum-is-very-slow-and-i-cant-figure-out-why/52791 "2021-01-03T19:15:11Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ferran\_Mazzanti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ferran_mazzanti/32/7458_2.png) [@Ferran\_Mazzanti](https://discourse.julialang.org/u/Ferran_Mazzanti)\
**Post date:** [January 3, 2021, 7:15pm UTC](https://discourse.julialang.org/t/sum-is-very-slow-and-i-cant-figure-out-why/52791/1 "2021-01-03T19:15:11Z")

</div>

Hi,  
I’m building up a Monte Carlo code in CUDA that, as intermediate steps, needs to perform sums of several thousand Floats. The fact is that the numbers are produced by a function V(r,dip) where r is vector of floats and dip is a parameter. The details of the function are irrelevant as I just work with the output it produces. I want to get sum of the vector of floats produced by V(r,dip). The nice thing is that, despite

```julia
@btime aux = V($r, $dip)
  33.156 μs (256 allocations: 7.03 KiB)

248000×1 CuArray{Float64,2}:
 1.8357571063480997
 0.4380869765616565
 0.960063385897547
 5.437484961363393
 0.27597788147989816
...

@btime CUDA.sum($aux)
  49.466 μs (68 allocations: 1.94 KiB)
260151.73723948127

```

if I put both things together in a function, times become crazily larger

```julia
function Sum_V(r,dip)
    aux = V(r,dip)
    CUDA.sum(aux)
end;
@btime Sum_V($r,$dip)
  5.385 ms (324 allocations: 8.97 KiB)
260151.73723948127

```

so I understand the problem of computer time increase is due to the allocation of the CuArray aux in the function? But I’m doing the very same operations separately above and the sum of times is less than 100 microseconds. So why this huge execution time penalty in Sum\_V?

I’ve tried also declaring aux externally, passing it as a parameter to Sum\_V, and replacing the `aux = V(r,dip)` with `aux .= V(r,dip)` with the same results.

BTW doing that on CPU takes 2.609ms, half the time it takes with my CUDA function -making it useless at this point.

Any help will be greatly appreciated 🙂

Ferran.

---

<div class="post-metadata">

**Author:** ![rveltz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rveltz/32/2707_2.png) [@rveltz](https://discourse.julialang.org/u/rveltz)\
**Post date:** [January 3, 2021, 7:35pm UTC](https://discourse.julialang.org/t/sum-is-very-slow-and-i-cant-figure-out-why/52791/2 "2021-01-03T19:35:37Z")

</div>

Try `CUDA.@time`, you are not measuring GPU computation time

---

<div class="post-metadata">

**Author:** ![Ferran\_Mazzanti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ferran_mazzanti/32/7458_2.png) [@Ferran\_Mazzanti](https://discourse.julialang.org/u/Ferran_Mazzanti)\
**Post date:** [January 3, 2021, 7:58pm UTC](https://discourse.julialang.org/t/sum-is-very-slow-and-i-cant-figure-out-why/52791/3 "2021-01-03T19:58:49Z")

</div>

Thanks, that makes sense…  
…but then my measured times are about the same as what I get on CPU and no CUDA (more or less 7ms), and I thought that would be much faster ☹

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [January 3, 2021, 8:01pm UTC](https://discourse.julialang.org/t/sum-is-very-slow-and-i-cant-figure-out-why/52791/4 "2021-01-03T20:01:57Z")

</div>

> [@Ferran\_Mazzanti](#):
>
> `CuArray{Float64,2}:`

This is a bad sign, lots of GPUs are much much slower with Float64, and mostly used with Float32.

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [January 4, 2021, 7:25am UTC](https://discourse.julialang.org/t/sum-is-very-slow-and-i-cant-figure-out-why/52791/5 "2021-01-04T07:25:42Z")

</div>

Most of the time is probably spent evaluating `aux = V($r, $dip)`, not during the `sum`, and by not doing `@btime CUDA.@sync` you’re not accurately measuring GPU execution time. The `sum` does, though, because you’re reducing to a single scalar which needs to be transferred to the CPU (hence this operation is implicitly synchronizing).