# Slow speed-up in simple GPU kernel

**URL:** <https://discourse.julialang.org/t/slow-speed-up-in-simple-gpu-kernel/70292>\
**Category:** New to Julia\
**Tags:** gpu\
**Created:** [October 24, 2021, 1:38pm UTC](https://discourse.julialang.org/t/slow-speed-up-in-simple-gpu-kernel/70292 "2021-10-24T13:38:02Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![adm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adm/32/18894_2.png) [@adm](https://discourse.julialang.org/u/adm)\
**Post date:** [October 24, 2021, 1:38pm UTC](https://discourse.julialang.org/t/slow-speed-up-in-simple-gpu-kernel/70292/1 "2021-10-24T13:38:02Z")

</div>

I am trying my first GPU kernel, which essentially boils down to the simplified functions below. I barely get a x1.7 speed up with respect to the multi-threaded version so I’m wondering whether/how the performance of the GPU version can be improved.

```julia
using CUDA

N = Int(2^20)
F = fill(false, N)
Q = fill(true, N)
d = rand(N)
Fd, Qd = CuArray(F), CuArray(Q)
dd = CuArray(d)
Δ = rand()

function update!(F, Q, dist, Δ)
    @inbounds for i in eachindex(F)
        F[i] = false
        if (Q[i] == true) && (dist[i] ≤ Δ)
            F[i] = true
        end
    end
end

function update_thread!(F, Q, dist, Δ)
    Threads.@threads for i in eachindex(F)
        @inbounds F[i] = false
        @inbounds if (Q[i] == true) && (dist[i] ≤ Δ)
            F[i] = true
        end
    end
end

function _gpu_update!(F, Q, dist, Δ)
    index = threadIdx().x 
    stride = blockDim().x
    @inbounds for i = index:stride:length(F)
        F[i] = false
        if (Q[i] == true) && (dist[i] ≤ Δ)
            F[i] = true
        end
    end
    return nothing
end

function gpu_update!(F::CuArray{Bool}, Q::CuArray{Bool}, dist::CuArray{T}, Δ) where T
    CUDA.@sync begin
        @cuda threads = 1024 _gpu_update!(F, Q, dist, Δ)
    end
end

@btime update!($F, $Q, $d, $Δ); # 4.062 ms (0 allocations: 0 bytes)
@btime update_thread!($F, $Q, $d, $Δ); # 1.005 ms (20 allocations: 1.94 KiB) on 4 threads
@btime gpu_update!($Fd, $Qd, $dd, $Δ); # 579.145 μs (179 allocations: 5.77 KiB)

```

---

<div class="post-metadata">

**Author:** ![jw3126](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jw3126/32/3086_2.png) [@jw3126](https://discourse.julialang.org/u/jw3126)\
**Post date:** [October 24, 2021, 5:11pm UTC](https://discourse.julialang.org/t/slow-speed-up-in-simple-gpu-kernel/70292/2 "2021-10-24T17:11:22Z")

</div>

You could try

```julia
F[i] = Q[i] & (dist[i] < Δ)

```

It is free of branches. Not sure if the compiler can eliminate the branches in your version.

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [October 25, 2021, 10:05am UTC](https://discourse.julialang.org/t/slow-speed-up-in-simple-gpu-kernel/70292/3 "2021-10-25T10:05:30Z")

</div>

The problem is that you’re only launching a single block. You need to launch multiple blocks, otherwise you’re essentially only using a single SM of your GPU.

---

<div class="post-metadata">

**Author:** ![adm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adm/32/18894_2.png) [@adm](https://discourse.julialang.org/u/adm)\
**Post date:** [October 25, 2021, 10:25am UTC](https://discourse.julialang.org/t/slow-speed-up-in-simple-gpu-kernel/70292/4 "2021-10-25T10:25:04Z")

</div>

You are right, now it works properly with a ~x66 speed up with respect to the single threaded function. Thanks

```julia
function _gpu_update!(F, Q, dist, Δ)
    index = blockIdx().x * blockDim().x + threadIdx().x
    stride = blockDim().x * gridDim().x
    @inbounds for i = index:stride:length(F)
        F[i] = false
        if (Q[i] == true) && (dist[i] ≤ Δ)
            F[i] = true
        end
    end
    return 
end

function gpu_update!(F::CuArray{Bool}, Q::CuArray{Bool}, dist::CuArray{T}, Δ) where T
    nt = 256
    numblocks = ceil(Int, (length(F) + nt -1)/nt)
    CUDA.@sync begin
        @cuda threads = nt blocks = numblocks _gpu_update!(F, Q, dist, Δ)
    end
end
```