# How to benchmark a function that uses KernelAbstractions kernels?

**URL:** <https://discourse.julialang.org/t/how-to-benchmark-a-function-that-uses-kernelabstractions-kernels/127026>\
**Category:** GPU\
**Tags:** question, kernelabstractions\
**Created:** [March 16, 2025, 9:17pm UTC](https://discourse.julialang.org/t/how-to-benchmark-a-function-that-uses-kernelabstractions-kernels/127026 "2025-03-16T21:17:29Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![pitsianis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pitsianis/32/26588_2.png) [@pitsianis](https://discourse.julialang.org/u/pitsianis)\
**Post date:** [March 16, 2025, 9:17pm UTC](https://discourse.julialang.org/t/how-to-benchmark-a-function-that-uses-kernelabstractions-kernels/127026/1 "2025-03-16T21:17:29Z")

</div>

What is the correct way to benchmark a function that uses `KernelAbstractions.jl` (KA) kernels?

I noticed that KA/CUDA i.e. KA with the `CUDA.jl` backend requires the `@synchronize(device)` macro.

Consider this

```julia
  @btime begin
    mykernel!($b)
    KernelAbstractions.synchronize($dev)
  end setup = (copyto!($b, $a0))
  @assert Array(b) == golden(a0)

```

where `a0` is a host array and `golden` the host function that performs a computation, `b` a device array and `mykernel` the device code attempting to do what `golden` does but faster.  
Is this the proper way to use `BenchmarkTools.jl` to measure time?

With the KA/Metal, the `@synchronize(device)` macro seems to add too much time, and without it, the performance times matched the expected. I did not manage to demonstrate the need for it by breaking correctness by removing it 🙂

For instance, this did not fail

```julia
using CUDA, KernelAbstractions

@kernel inbounds=true function addone!(a)
    i = @index(Global, Linear)
    a[i] += 1
end

function test(n)
  a = CuArray(rand(Float32, n,1))
  c = copy(a) .+ 1
  dev = get_backend(a)

  kernel = addone!(dev, 1024)

  for i = 1:100000
    kernel(a, ndrange=n)
    if !all(a .== c)
      @warn "Failure!"
    end
    c .+= 1
  end
end

test(2^25)

```

The main idea was that device array `c` has the correct value that device array `a` will get only after each `kernel` invocation is completed. I ran it for 100K times and no failure. Why?

---

<div class="post-metadata">

**Author:** ![yolhan\_mannes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yolhan_mannes/32/220485_2.png) [@yolhan\_mannes](https://discourse.julialang.org/u/yolhan_mannes)\
**Post date:** [March 17, 2025, 5:32am UTC](https://discourse.julialang.org/t/how-to-benchmark-a-function-that-uses-kernelabstractions-kernels/127026/2 "2025-03-17T05:32:29Z")

</div>

Maybe the .== Is also making a kernel that make sure everything is synchronise before and after it’s called? You could try to make your own reduction kernel for the !all(a.==c) and try

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [March 17, 2025, 8:12am UTC](https://discourse.julialang.org/t/how-to-benchmark-a-function-that-uses-kernelabstractions-kernels/127026/3 "2025-03-17T08:12:20Z")

</div>

You always need to synchronize, regardless of the back-end. Semantically, kernel launches are asynchronous, so you’re benchmarking launch time instead of execution time without synchronization. If this introduces performance problems with Metal.jl, please file an issue there.

With respect to your second example: `all` calls `mapreduce`, which automatically synchronizes. Generally, you won’t be able to observe the asynchronous nature of GPU operations when using array abstractions, as they ought to automatically synchronize when needed.

---

<div class="post-metadata">

**Author:** ![pitsianis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pitsianis/32/26588_2.png) [@pitsianis](https://discourse.julialang.org/u/pitsianis)\
**Post date:** [March 17, 2025, 9:57am UTC](https://discourse.julialang.org/t/how-to-benchmark-a-function-that-uses-kernelabstractions-kernels/127026/4 "2025-03-17T09:57:49Z")

</div>

> [@yolhan\_mannes](#):
>
> You could try to make your own reduction kernel for the !all(a.==c) and try

I guess if I were to call the checker in a different stream, it would catch the device array before the change.

Is there the abstraction of streams in KA and Metal like in CUDA? Sequences of operations (such as kernel launches, memory copies, and synchronization commands) submitted to the same stream, execute in order.

How do I submit the `!all` custom kernel to a different stream in KA or in Metal?

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [March 17, 2025, 1:27pm UTC](https://discourse.julialang.org/t/how-to-benchmark-a-function-that-uses-kernelabstractions-kernels/127026/5 "2025-03-17T13:27:19Z")

</div>

> [@pitsianis](#):
>
> How do I submit the `!all` custom kernel to a different stream in KA or in Metal?

Use a different Julia task.

> [@pitsianis](#):
>
> I guess if I were to call the checker in a different stream, it would catch the device array before the change.

CUDA.jl automatically synchronizes when switching tasks, so you won’t catch it.