# GPU Synchronization Issue - using KernelAbstraction

**URL:** <https://discourse.julialang.org/t/gpu-synchronization-issue-using-kernelabstraction/106601>\
**Category:** GPU\
**Tags:** question\
**Created:** [November 22, 2023, 10:05pm UTC](https://discourse.julialang.org/t/gpu-synchronization-issue-using-kernelabstraction/106601 "2023-11-22T22:05:14Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![erasedbird](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/erasedbird/32/202739_2.png) [@erasedbird](https://discourse.julialang.org/u/erasedbird)\
**Post date:** [November 22, 2023, 10:05pm UTC](https://discourse.julialang.org/t/gpu-synchronization-issue-using-kernelabstraction/106601/1 "2023-11-22T22:05:14Z")

</div>

@synchronize is not working consistently for this function I am trying to write, here is what is happening. This function is supposed to first divide everything in the first column by the topmost element of column 1 (3), then synchronizes, then copies the second and third elements of column 1 into the second and third elements of columns 2 and 3.

About half the time I run it the correct answer (a matrix where the second column is 1 2 3) comes out, other half of the time incorrect answer (matrix where second column is 1 6 9) comes out. However, it always copies column 3 correctly.

Does anyone know what is happening and how I can fix this? Thank you!

![Screenshot 2023-11-22 at 5.04.33 PM](https://global.discourse-cdn.com/julialang/original/3X/e/5/e56dc55287ea6429bc9c47cb428c39c5fb5561d4.png)

```julia
A = CuArray([3.0 1.0 1.0; 6.0 4.0 2.0; 9.0 7.0 7.0])
backend = get_backend(A)
@kernel function test_gpu!(A)
    I, J = @index(Global, NTuple)
        
    if I <= 2 && J == 1
        A[I+1, 1] = A[I+1, 1]/A[1, 1]
        
    end
    @synchronize
        
        
    if I > 1 && I <= 3 && J <= 3 && J > 1
        A[I, J] = A[I, 1]
    end
    @synchronize

   
end

test_gpu!(backend, 64)(A, ndrange = (3,3))
A

```

(Also, happy Thanksgiving to everyone celebrating!)

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [November 24, 2023, 8:13am UTC](https://discourse.julialang.org/t/gpu-synchronization-issue-using-kernelabstraction/106601/2 "2023-11-24T08:13:58Z")

</div>

KA.jl is launching multiple blocks here, and `@synchronize` (aka. `CUDA.sync_threads()`) only synchronizes threads within a block.

---

<div class="post-metadata">

**Author:** ![erasedbird](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/erasedbird/32/202739_2.png) [@erasedbird](https://discourse.julialang.org/u/erasedbird)\
**Post date:** [November 24, 2023, 9:22pm UTC](https://discourse.julialang.org/t/gpu-synchronization-issue-using-kernelabstraction/106601/3 "2023-11-24T21:22:45Z")

</div>

Thank you for the explanation! Do you know if there’s a way to make KA launch just one block at a time? I couldn’t find anything in the documentation.

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [November 25, 2023, 8:20am UTC](https://discourse.julialang.org/t/gpu-synchronization-issue-using-kernelabstraction/106601/4 "2023-11-25T08:20:18Z")

</div>

If your workgroup size matches the ndrange, you will only get a single block. So pass `(3,3)` instead of `64`.

---

<div class="post-metadata">

**Author:** ![erasedbird](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/erasedbird/32/202739_2.png) [@erasedbird](https://discourse.julialang.org/u/erasedbird)\
**Post date:** [November 25, 2023, 4:38pm UTC](https://discourse.julialang.org/t/gpu-synchronization-issue-using-kernelabstraction/106601/5 "2023-11-25T16:38:45Z")

</div>

This worked, thank you so much!

---

<div class="post-metadata">

**Author:** ![evelyne-ringoot](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/evelyne-ringoot/32/45001_2.png) [@evelyne-ringoot](https://discourse.julialang.org/u/evelyne-ringoot)\
**Post date:** [December 13, 2023, 8:51pm UTC](https://discourse.julialang.org/t/gpu-synchronization-issue-using-kernelabstraction/106601/6 "2023-12-13T20:51:30Z")

</div>

Hi, thanks for the question and answer above! Is the original code:

> test\_gpu!(backend, 64)(A, ndrange = (3,3))

launching 3 workgroups of size 3x1?