# Using stream per cpu thread pattern

**URL:** <https://discourse.julialang.org/t/using-stream-per-cpu-thread-pattern/25053>\
**Category:** GPU\
**Created:** [June 7, 2019, 5:49pm UTC](https://discourse.julialang.org/t/using-stream-per-cpu-thread-pattern/25053 "2019-06-07T17:49:44Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alex\_Ellison](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alex_ellison/32/5749_2.png) [@Alex\_Ellison](https://discourse.julialang.org/u/Alex_Ellison)\
**Post date:** [June 7, 2019, 5:49pm UTC](https://discourse.julialang.org/t/using-stream-per-cpu-thread-pattern/25053/1 "2019-06-07T17:49:44Z")

</div>

This CUDA tutorial demonstrates how multiple cpu threads can be spawned and each will have its own stream, which achieves great concurrency on the GPU (see section " A Multi-threading Example") [GPU Pro Tip: CUDA 7 Streams Simplify Concurrency | NVIDIA Technical Blog](https://devblogs.nvidia.com/gpu-pro-tip-cuda-7-streams-simplify-concurrency/) . Ultimately, I’d like to have a multi-threaded program where each thread can spawn kernels on its own stream, and can sync/wait for them.

I’m trying to replicate this by modifying the MWE from this other active thread [CUDA streams do not overlap](https://discourse.julialang.org/t/cuda-streams-do-not-overlap/24879). I’m just adding

My code and output are:

```julia
using CUDAdrv, CUDAnative, CuArrays

function memcopy!(A, B)
    ix = (blockIdx().x-1) * blockDim().x + threadIdx().x
    A[ix] = B[ix]
    return
end

function main()
    nx = 128*1024^2
    nt = 100
    nthreads = 1024
    nblocks = ceil(Int, nx/nthreads)
    Threads.@threads for i = 1:2
        A = CuArray(zeros(nx))
        B = CuArray(ones(nx))
        s = CuStream()
        @cuda blocks=nblocks threads=nthreads stream=s memcopy!(A, B);
        CUDAdrv.synchronize(s)
    end
end

main()

```

Which, only with the `Threads.@threads`, produces:

```julia
Error thrown in threaded loop on thread 1: CUDAdrv.CuError(code=201, meta=nothing)
julia>

```

Without multiple threads, I have no problem. Is there a better way I should be either multithreading or using the CUDAdrv/native packages?

Thanks!

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [June 8, 2019, 9:08pm UTC](https://discourse.julialang.org/t/using-stream-per-cpu-thread-pattern/25053/2 "2019-06-08T21:08:33Z")

</div>

This is currently no possible, Julia’s threading support is experimental and the combination with CUDA is not something that currently works (as far as I know).