# Profiling Julia CUDA code missing 'CUDA HW'

**URL:** https://discourse.julialang.org/t/profiling-julia-cuda-code-missing-cuda-hw/76061
**Category:** GPU
**Created:** [February 9, 2022, 7:18am UTC](https://discourse.julialang.org/t/profiling-julia-cuda-code-missing-cuda-hw/76061 "2022-02-09T07:18:46Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![torrance](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torrance/32/38990_2.png) [@torrance](https://discourse.julialang.org/u/torrance)
#### Post date: [February 9, 2022, 7:18am UTC](https://discourse.julialang.org/t/profiling-julia-cuda-code-missing-cuda-hw/76061/1 "2022-02-09T07:18:46Z")

</div>

I’m attempting to profile a custom CUDA kernel using NVIDIA’s nsight-systems (ie. nsys), as recommended by the CUDA.jl project.

nsys correctly gives me profiling results for nvcc compiled code, however when I run it against Julia, I only see CPU thread entries. It is missing the _CUDA HW_ entry in the timeline where I am expecting to see information about the kernel execution, memory copies, etc.

Has anyone else experienced this, and is there a fix?

---

<div class="post-metadata">

### Author: ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)
#### Post date: [February 9, 2022, 8:30am UTC](https://discourse.julialang.org/t/profiling-julia-cuda-code-missing-cuda-hw/76061/2 "2022-02-09T08:30:10Z")

</div>

Are you actually profiling GPU code? Can you try with a simple `CUDA.@profile CuArray([1]) .+ 1` or so?

---

<div class="post-metadata">

### Author: ![torrance](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torrance/32/38990_2.png) [@torrance](https://discourse.julialang.org/u/torrance)
#### Post date: [February 9, 2022, 8:47am UTC](https://discourse.julialang.org/t/profiling-julia-cuda-code-missing-cuda-hw/76061/3 "2022-02-09T08:47:10Z")

</div>

Yes I believe I am. I’ve reduced the minimum code to this:

```
nsys profile julia -e 'using CUDA; arr = rand(100); arrd = CuArray(arr); map(arrd) do x; sin(x); end; println(Array(arrd))'

```

Actually, if I force `--cuda-memory-usage=true`, I do in fact get a “CUDA HW” header, but it does not include any information about the CUDA threads. See attached screenshot for an example.

 ![Screenshot 2022-02-09 164643](https://global.discourse-cdn.com/julialang/original/3X/e/1/e1cedf1ce76a73f5106c29f8560240b98aa41c02.png)

---

<div class="post-metadata">

### Author: ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)
#### Post date: [February 9, 2022, 8:48am UTC](https://discourse.julialang.org/t/profiling-julia-cuda-code-missing-cuda-hw/76061/4 "2022-02-09T08:48:33Z")

</div>

Strange. Are there actual CUDA API calls in there? Can you share the .nsys-rep file?

---

<div class="post-metadata">

### Author: ![torrance](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torrance/32/38990_2.png) [@torrance](https://discourse.julialang.org/u/torrance)
#### Post date: [February 9, 2022, 8:53am UTC](https://discourse.julialang.org/t/profiling-julia-cuda-code-missing-cuda-hw/76061/5 "2022-02-09T08:53:24Z")

</div>

Sure please see the .nsys-rep file here: [Sign in to your account](https://curtin-my.sharepoint.com/:u:/g/personal/277966k_curtin_edu_au/ETHAh1saleNCt3ONCA_6gRABj1aLQCR9cWGqBO8hEml0iA?e=ykjfbZ)

---

<div class="post-metadata">

### Author: ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)
#### Post date: [February 9, 2022, 9:01am UTC](https://discourse.julialang.org/t/profiling-julia-cuda-code-missing-cuda-hw/76061/6 "2022-02-09T09:01:07Z")

</div>

Kernel launches are definitely there, but the `CUDA profiling might have not been started correctly.` is suspicious. Maybe try upgrading to the latest NSight.

---

<div class="post-metadata">

### Author: ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)
#### Post date: [February 9, 2022, 9:17am UTC](https://discourse.julialang.org/t/profiling-julia-cuda-code-missing-cuda-hw/76061/7 "2022-02-09T09:17:30Z")

</div>

Also, which version of CUDA.jl? Could you post the output of running under JULIA\_DEBUG=CUDA?

---

<div class="post-metadata">

### Author: ![torrance](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torrance/32/38990_2.png) [@torrance](https://discourse.julialang.org/u/torrance)
#### Post date: [February 9, 2022, 9:46am UTC](https://discourse.julialang.org/t/profiling-julia-cuda-code-missing-cuda-hw/76061/8 "2022-02-09T09:46:17Z")

</div>

I can confirm its happening with other non-Julia code. So this would appear to be a system issue, rather than Julia specific. I’ll try to get one of the HPC staff onto it.

Thanks for your help @maleadt Tim!
