# Can CUDA beat LoopVectorization/make this CUDA code faster?

**URL:** <https://discourse.julialang.org/t/can-cuda-beat-loopvectorization-make-this-cuda-code-faster/98804>\
**Category:** General Usage\
**Tags:** question\
**Created:** [May 13, 2023, 6:38pm UTC](https://discourse.julialang.org/t/can-cuda-beat-loopvectorization-make-this-cuda-code-faster/98804 "2023-05-13T18:38:36Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![jw3126](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jw3126/32/3086_2.png) [@jw3126](https://discourse.julialang.org/u/jw3126)\
**Post date:** [May 13, 2023, 6:38pm UTC](https://discourse.julialang.org/t/can-cuda-beat-loopvectorization-make-this-cuda-code-faster/98804/1 "2023-05-13T18:38:36Z")

</div>

I have the following code, which has very nice performance on the CPU:

```julia
using LoopVectorization: @turbo

function dilation_loop_vec!(out_cpu, inp_cpu, ker_cpu)
    @turbo thread=true for i in eachindex(out_cpu)
        val = typemin(eltype(out_cpu))
        for k in eachindex(ker_cpu)
            val = max(val, (inp_cpu[i-1+k] + ker_cpu[k]))
        end
        out_cpu[i] = val
    end
    return out_cpu
end

```

How to implement it in CUDA.jl in a fast way? I tried the following:

> ****
>
> ```julia
> using CUDA
> 
> function dilation_cuda_kernel_naive!(
> out::AbstractVector, 
> inp::AbstractVector, 
> ker::AbstractVector)
> index = (blockIdx().x - 1) * blockDim().x + threadIdx().x
> stride = gridDim().x * blockDim().x
> for iout = index:stride:length(out)
> val = typemin(eltype(out))
> for iker in eachindex(ker)
> @inbounds val = max(val, (inp[iout + iker-1] + ker[iker]))
> end
> @inbounds out[iout] = val
> end
> return nothing
> end
> 
> function dilation_cuda_naive!(out_gpu, inp_gpu, ker_gpu)
> kernel = CUDA.@cuda launch=false dilation_cuda_kernel_naive!(out_gpu, inp_gpu, ker_gpu)
> config = CUDA.launch_configuration(kernel.fun)
> threads = min(length(out_gpu), config.threads)
> blocks = cld(length(out_gpu), threads)
> CUDA.@sync begin
> kernel(out_gpu, inp_gpu, ker_gpu; threads, blocks)
> end
> return out_gpu
> end
> 
> ```

Here are the results, LoopVectorization blows my naive CUDA version out of the water.

```julia
T = Float32
# T = UInt16
out_cpu = rand(T, 2^20)
ker_cpu = rand(T, 2^14)
inp_cpu = rand(T, length(out_cpu) + length(ker_cpu) - 1)
out_gpu = CuArray(out_cpu)
ker_gpu = CuArray(ker_cpu)
inp_gpu = CuArray(inp_cpu)

res_gpu = dilation_cuda_naive!(out_gpu, inp_gpu, ker_gpu)
res_cpu = dilation_loop_vec!(out_cpu, inp_cpu, ker_cpu)
@assert res_cpu ≈ Array(res_gpu)
@showtime dilation_cuda_naive!(out_gpu, inp_gpu, ker_gpu)
@showtime dilation_loop_vec!(out_cpu, inp_cpu, ker_cpu)
nothing

```

```julia
dilation_cuda_naive!(out_gpu, inp_gpu, ker_gpu): 0.134039 seconds (60 allocations: 3.516 KiB)
dilation_loop_vec!(out_cpu, inp_cpu, ker_cpu): 0.050131 seconds (3 allocations: 96 bytes)

```