# Parallelizaton on GPU slower than on CPU...?

**URL:** <https://discourse.julialang.org/t/parallelizaton-on-gpu-slower-than-on-cpu/33587>\
**Category:** Performance\
**Tags:** gpu\
**Created:** [January 20, 2020, 2:17pm UTC](https://discourse.julialang.org/t/parallelizaton-on-gpu-slower-than-on-cpu/33587 "2020-01-20T14:17:08Z")\
**Posts on this page:** 1\
**Showing post:** 5

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [January 21, 2020, 12:54am UTC](https://discourse.julialang.org/t/parallelizaton-on-gpu-slower-than-on-cpu/33587/5 "2020-01-21T00:54:44Z")

</div>

Those are some impressive numbers on the 1950X!

Note that even though the two arrays only take up 16 MiB (2^21 \* 4 \* 2 / 2^20 = 2^4), the computation is memory bound.

```julia
julia> N = 2^21
2097152

julia> flops = 10^6 * N / 44.499
4.712807029371446e10

```

I don’t know what clock speed your CPU runs at all-core, so I’ll pick 4 GHz:

```julia
julia> Hz = 4e9; fma_per_clock = 2; flop_per_fma = 16; cores = 16;

julia> Hz * fma_per_clock * flop_per_fma * cores
2.048e12

julia> ans / flops
43.4560546875

```

Your CPU was mostly sitting, waiting for data. For every nanosecond it spent computing, there were 40 doing nothing.

For comparison, on my 10980XE, my sequential and parallel times were 705 and 58 microseconds.  
Thus, my numbers are

```julia
julia> Hz = 4.1e9; fma_per_clock = 2; flop_per_fma = 32; cores = 18;

julia> Hz * fma_per_clock * flop_per_fma * cores
4.7232e12

julia> ans / (10^6 * N / 58)
130.62744140625

```

Yikes. My ratio was about 130.

I don’t know much about GPU computing, but I bet you couldn’t bring it’s number crunching power to bear. Longer vectors would just make the memory problems worse.

I also don’t enough yet about memory to say anything about TLB misses vs memory bandwidth, but I’ll start looking into that sort of thing one day.

For memory bound operations, memory performance dominates. Regardless of the reason, the Ryzen 3950X looks amazing here.

---

_[View the full topic](https://discourse.julialang.org/t/parallelizaton-on-gpu-slower-than-on-cpu/33587)._