# CUDA async is not working properly

**URL:** <https://discourse.julialang.org/t/cuda-async-is-not-working-properly/123902>\
**Category:** New to Julia\
**Tags:** gpu, cuda\
**Created:** [December 16, 2024, 9:57pm UTC](https://discourse.julialang.org/t/cuda-async-is-not-working-properly/123902 "2024-12-16T21:57:06Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![miniskar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/miniskar/32/210144_2.png) [@miniskar](https://discourse.julialang.org/u/miniskar)\
**Post date:** [December 16, 2024, 9:57pm UTC](https://discourse.julialang.org/t/cuda-async-is-not-working-properly/123902/1 "2024-12-16T21:57:06Z")

</div>

I am trying to run saxpy\_kernel using CUDA async. I have a need to use the user specified stream to run the saxpy\_kernel. It is working good with CUDA.sync, but with CUDA.@async.

Here is the minimum working example.  
File: stream\_pure.jl

```julia
using CUDA

# Define the SAXPY kernel
function saxpy_kernel(z, a, x, y)
    i = threadIdx().x + (blockIdx().x - 1) * blockDim().x
    if i <= length(x)
        z[i] = a * x[i] + y[i]
    end
    return
end

# Main function demonstrating CUDA.@async
function main()
    # Array size
    n = 8
    # Create random input data
    x = CUDA.rand(Float32, n)
    y = CUDA.rand(Float32, n)
    z = CUDA.zeros(Float32, n)
    a = 2.5f0

    # Create CUDA streams
    stream = CUDA.CuStream()

    # Launch SAXPY in parallel across multiple streams
    CUDA.@async begin
            println("Launching task on stream")
            @cuda stream=stream threads=256 blocks=ceil(Int, length(x) / 256) saxpy_kernel(z, a, x, y)
            synchronize(stream) # Ensure the stream completes
            println("Task completed.")
    end

    z_host = Array(z)
    println("All tasks completed.", z_host)
end

# Run the main function
main()

```

How to run:

```julia
$ julia stream_pure.jl

```

Here is the outcome:

```julia
All tasks completed.Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]

```

Expected outcome:  
all non zeros after completed message.

---

<div class="post-metadata">

**Author:** ![contradict](https://avatars.discourse-cdn.com/v4/letter/c/ac91a4/32.png) [@contradict](https://discourse.julialang.org/u/contradict)\
**Post date:** [December 16, 2024, 11:23pm UTC](https://discourse.julialang.org/t/cuda-async-is-not-working-properly/123902/2 "2024-12-16T23:23:53Z")

</div>

You are not waiting for the task you launched with `@async` to complete. `CUDA.@async` is the same as [`Base.@async`](https://docs.julialang.org/en/v1/base/parallel/#Base.@async) and has nothing to do with CUDA or GPU programming, methods from `Base` are present in all modules. You could [`wait()`](https://docs.julialang.org/en/v1/base/parallel/#Base.wait) for that task, but unless you have other tasks to run on the same thread, you should leave out the `@async` call. The stream argument to `@cuda` and the subsequent synchronize do everything you need.

---

<div class="post-metadata">

**Author:** ![miniskar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/miniskar/32/210144_2.png) [@miniskar](https://discourse.julialang.org/u/miniskar)\
**Post date:** [December 17, 2024, 12:56am UTC](https://discourse.julialang.org/t/cuda-async-is-not-working-properly/123902/3 "2024-12-17T00:56:27Z")

</div>

Thank you. It helped to resolve this issue.

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [December 17, 2024, 6:05am UTC](https://discourse.julialang.org/t/cuda-async-is-not-working-properly/123902/4 "2024-12-17T06:05:13Z")

</div>

In addition, it may be better to use `CUDA.stream!` to switch the stream of the current task (either temporarily using `do`-block syntax, or permanently). If you only pass `stream` to `@cuda`, other operations will still use the default task-local stream. If you’d switch instead, you should be able to copy by calling `Array` without having to `synchronize`. Of course, this may not apply to your actual application, but it does to this MWE.

---

<div class="post-metadata">

**Author:** ![miniskar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/miniskar/32/210144_2.png) [@miniskar](https://discourse.julialang.org/u/miniskar)\
**Post date:** [December 31, 2024, 2:18pm UTC](https://discourse.julialang.org/t/cuda-async-is-not-working-properly/123902/5 "2024-12-31T14:18:21Z")

</div>

Understood. My need is also to use the stream only the specified task. Thank you for the replies.