# Asynchronous kernel scheduling with KernelAbstractions
**URL:** https://discourse.julialang.org/t/asynchronous-kernel-scheduling-with-kernelabstractions/100535
**Category:** GPU
**Created:** [June 18, 2023, 11:32pm UTC](https://discourse.julialang.org/t/asynchronous-kernel-scheduling-with-kernelabstractions/100535 "2023-06-18T23:32:21Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@smartalecH](https://discourse.julialang.org/u/smartalecH)
#### Post date: [June 18, 2023, 11:32pm UTC](https://discourse.julialang.org/t/asynchronous-kernel-scheduling-with-kernelabstractions/100535/1 "2023-06-18T23:32:21Z")
The KernelAbstractions docs mention that kernels are launched asynchronously. I’m hoping to leverage this within a solver I’m working on, where I hope to _hide_ the communication between GPUs behind some computation (a common technique with finite difference codes).
Typically, I would have to map different kernel calls to different SMs myself (if using CUDA). Does KernelAbstractions do this under the hood (for the various backends that support this)? Or are there some “scheduling implications” I should be aware of?
Thanks!
---
### Author:  [@maleadt](https://discourse.julialang.org/u/maleadt)
#### Post date: [June 19, 2023, 6:06am UTC](https://discourse.julialang.org/t/asynchronous-kernel-scheduling-with-kernelabstractions/100535/2 "2023-06-19T06:06:20Z")
> [@smartalecH](#):
>
> map different kernel calls to different SMs
Can you elaborate? With CUDA, you cannot decide which SMs a kernel executes on. That would also only matter if you want to overlap kernel execution, which is separate from their asynchronous nature.
---
### Author:  [@smartalecH](https://discourse.julialang.org/u/smartalecH)
#### Post date: [June 19, 2023, 2:36pm UTC](https://discourse.julialang.org/t/asynchronous-kernel-scheduling-with-kernelabstractions/100535/3 "2023-06-19T14:36:32Z")
Ah, thanks @maleadt. You’re right, I should clarify.
What I’m really after is overlapping kernels, and not just asynchronous launching.
After a bit more digging, it seems like there’s been some work around this within KernelAbstractions, but I’m not sure what the current status is:
>
>
> KA currently uses a very verbose and explicit dependency management.
>
> \`\`\`
> eve…nt = kernel(CPU())(...)
> event = kernel(CPU())(..., dependencies=(event,))
> \`\`\`
>
> This was added since at the time CUDA.jl used one stream, and thus exposing concurrency was harder.
>
> Now @maleadt added a really nice design around task local streams, allowing users to use Julia tasks to express concurrency on the GPU as well.
>
> So I am thinking that in the interest of reducing the complexity of KA in usage and to align it better with CUDA.jl I would like to remove the dependency management
> and move to a stream based model.
>
> One open question is how to deal with the CPU (but this could mean we simply move to synchronous execution there, reducing latency as well)
>
> An alternative that I see is to explore an more implicit dependency model based on the arguments to the kernel, I think that would be similar to SYCL or what AMDGPU currently does.
>
> This would be the first step towards KA 1.0
>
> CC interested parties: @glwagner @lcw @jpsamaroo @simonbyrne @kpamnany @omlins
---
### Author:  [@maleadt](https://discourse.julialang.org/u/maleadt)
#### Post date: [June 19, 2023, 4:59pm UTC](https://discourse.julialang.org/t/asynchronous-kernel-scheduling-with-kernelabstractions/100535/4 "2023-06-19T16:59:01Z")
After [https://github.com/JuliaGPU/KernelAbstractions.jl/pull/317](https://github.com/JuliaGPU/KernelAbstractions.jl/pull/317), KA.jl should be compatible with CUDA.jl’s task mechanism. So you should use Julia tasks in order for kernels to launch on different streams, and potentially overlap. See [CUDA.jl 3.0 ⋅ JuliaGPU](https://juliagpu.org/post/2021-04-09-cuda_3.0/#improved_multi-tasking_and_multi-threading)
---
### Author:  [@vchuravy](https://discourse.julialang.org/u/vchuravy)
#### Post date: [June 23, 2023, 10:36pm UTC](https://discourse.julialang.org/t/asynchronous-kernel-scheduling-with-kernelabstractions/100535/5 "2023-06-23T22:36:20Z")
In particular with KernelAbstractions 0.9 you would do it “just like” CUDA.jl
You can use multiple Julia tasks to represent concurrent work and example here is
>
where I use Julia tasks to do some MPI communication concurrently.
---
### Author:  [@smartalecH](https://discourse.julialang.org/u/smartalecH)
#### Post date: [June 25, 2023, 4:53am UTC](https://discourse.julialang.org/t/asynchronous-kernel-scheduling-with-kernelabstractions/100535/6 "2023-06-25T04:53:21Z")
Thank you @maleadt and @vchuravy! This helps a lot.