# GPU

**URL:** https://discourse.julialang.org/c/domain/gpu/11.md?page=13

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 14

---

## [Advice for optimising code for GPU](https://discourse.julialang.org/t/advice-for-optimising-code-for-gpu/95939)

<div class="topic-metadata">

**Author:** [@maaaks](https://discourse.julialang.org/u/maaaks)\
**Replies:** 3\
**Last updated:** [March 13, 2023, 10:39am UTC](https://discourse.julialang.org/t/advice-for-optimising-code-for-gpu/95939 "2023-03-13T10:39:34Z")

</div>

Hello! I have recently ported a matlab package to Julia (and found the process suprisingly straight forward), however I with to be able to execute it on the GPU as I with to use it as part of a loss function in an ML pr…

---

## [How do I avoid atomics in this kind of code on GPU?](https://discourse.julialang.org/t/how-do-i-avoid-atomics-in-this-kind-of-code-on-gpu/95881)

<div class="topic-metadata">

**Author:** [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Replies:** 2\
**Last updated:** [March 11, 2023, 8:39am UTC](https://discourse.julialang.org/t/how-do-i-avoid-atomics-in-this-kind-of-code-on-gpu/95881 "2023-03-11T08:39:14Z")

</div>

I want to stress that this is a toy example. It is not a real calculation. It is a toy example, intended to be a minimal working example. It is showing some approach I want to apply to a problem of mine Hello! I am wri…

---

## [sharedMemory in GPU programming examples](https://discourse.julialang.org/t/sharedmemory-in-gpu-programming-examples/95567)

<div class="topic-metadata">

**Author:** [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Replies:** 3\
**Last updated:** [March 7, 2023, 6:32pm UTC](https://discourse.julialang.org/t/sharedmemory-in-gpu-programming-examples/95567 "2023-03-07T18:32:49Z")

</div>

Hello! I found this great ressource here: https://jenni-westoby.github.io/Julia\_GPU\_examples/dev/Vector\_dot\_product/ And am going through the motion of understanding this. In the mean while I hope some could provide m…

---

## [CUDA atomic add for ForwardDiff duals?](https://discourse.julialang.org/t/cuda-atomic-add-for-forwarddiff-duals/85968)

<div class="topic-metadata">

**Author:** [@Mark\_Lau](https://discourse.julialang.org/u/Mark_Lau)\
**Replies:** 2\
**Last updated:** [August 19, 2022, 5:59pm UTC](https://discourse.julialang.org/t/cuda-atomic-add-for-forwarddiff-duals/85968 "2022-08-19T17:59:41Z")

</div>

Hi everyone, I’ve been trying to auto differentiate some code that runs on the GPU using ForwardDiff. Some of the kernels perform an atomic addition operation via CUDA.@atomic, but that doesn’t seem to be supported for F…

---

## [CuPy CuFFT ~2x faster than CUDA.jl CuFFT](https://discourse.julialang.org/t/cupy-cufft-2x-faster-than-cuda-jl-cufft/89151)

<div class="topic-metadata">

**Author:** [@Dreycen\_Foiles](https://discourse.julialang.org/u/Dreycen_Foiles)\
**Replies:** 15\
**Last updated:** [February 27, 2023, 6:42pm UTC](https://discourse.julialang.org/t/cupy-cufft-2x-faster-than-cuda-jl-cufft/89151 "2023-02-27T18:42:31Z")

</div>

I am working on a simulation whose bottleneck is lots of FFT-based convolutions performed on the GPU. I wanted to see how FFT’s from CUDA.jl would compare with one of bigger Python GPU libraries CuPy. I was surprised to …

---

## [Bitwise operator "&" won't work in kernel functions](https://discourse.julialang.org/t/bitwise-operator-wont-work-in-kernel-functions/93159)

<div class="topic-metadata">

**Author:** [@zero](https://discourse.julialang.org/u/zero)\
**Replies:** 2\
**Last updated:** [February 23, 2023, 2:26pm UTC](https://discourse.julialang.org/t/bitwise-operator-wont-work-in-kernel-functions/93159 "2023-02-23T14:26:56Z")

</div>

Hi everyone, CUDA does not have an intrinsic function to operate “&”. However, in c/c++ you can still use the intrinsic bitwise \_\_and() to get the same result in your kernel method, or in python for example by cupy you …

---

## [This example in CUDA.jl does not work for me?](https://discourse.julialang.org/t/this-example-in-cuda-jl-does-not-work-for-me/94294)

<div class="topic-metadata">

**Author:** [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Replies:** 5\
**Last updated:** [February 17, 2023, 11:23pm UTC](https://discourse.julialang.org/t/this-example-in-cuda-jl-does-not-work-for-me/94294 "2023-02-17T23:23:12Z")

</div>

https://cuda.juliagpu.org/stable/api/kernel/#Element-access-and-broadcasting using Test using CUDA a = rand(Float16, (16, 16)) b = rand(Float16, (16, 16)) c = rand(Float32, (16, 16)) a\_dev = CuArray(a) b\_…

---

## [Why is 'trace' slow on CUDA matrices?](https://discourse.julialang.org/t/why-is-trace-slow-on-cuda-matrices/94723)

<div class="topic-metadata">

**Author:** [@HenriDeh](https://discourse.julialang.org/u/HenriDeh)\
**Replies:** 4\
**Last updated:** [February 16, 2023, 4:05pm UTC](https://discourse.julialang.org/t/why-is-trace-slow-on-cuda-matrices/94723 "2023-02-16T16:05:09Z")

</div>

Hello, I am kind of surprised to see this benchmark: julia\> using BenchmarkTools, CUDA, LinearAlgebra julia\> A = rand(128, 128); julia\> Ad = cu(A); julia\> @btime tr($A) 70.092 ns (0 allocations: 0 bytes) 64.994448…

---

## [CUDA.jl CUDA error 803](https://discourse.julialang.org/t/cuda-jl-cuda-error-803/94631)

<div class="topic-metadata">

**Author:** [@Kieran\_Marray](https://discourse.julialang.org/u/Kieran_Marray)\
**Replies:** 2\
**Last updated:** [February 15, 2023, 8:50am UTC](https://discourse.julialang.org/t/cuda-jl-cuda-error-803/94631 "2023-02-15T08:50:13Z")

</div>

Hi Have been trying to install CUDA.jl for the first time on Ubuntu 22.04 with a NVIDIA Quadro 400 GPU. I downloaded the correct drivers from nvidia (the 525 drivers), and julia from the main julia page as opposed to sn…

---

## [CUDA errors 203 and 103, CUDA.rand() and CUDA.versioninfo()](https://discourse.julialang.org/t/cuda-errors-203-and-103-cuda-rand-and-cuda-versioninfo/94595)

<div class="topic-metadata">

**Author:** [@David\_A](https://discourse.julialang.org/u/David_A)\
**Replies:** 6\
**Last updated:** [February 15, 2023, 7:09am UTC](https://discourse.julialang.org/t/cuda-errors-203-and-103-cuda-rand-and-cuda-versioninfo/94595 "2023-02-15T07:09:02Z")

</div>

I get the below error when attempting to use CUDa.rand() julia\> using CUDA julia\> CUDA.rand() ERROR: CURANDError: initialization of CUDA failed (code 203, CURAND\_STATUS\_INITIALIZATION\_FAILED) Stacktrace: \[1\] throw\_…

---

## [Broadcast over CuArray error](https://discourse.julialang.org/t/broadcast-over-cuarray-error/94654)

<div class="topic-metadata">

**Author:** [@jinfreedom](https://discourse.julialang.org/u/jinfreedom)\
**Replies:** 3\
**Last updated:** [February 15, 2023, 1:05am UTC](https://discourse.julialang.org/t/broadcast-over-cuarray-error/94654 "2023-02-15T01:05:40Z")

</div>

Hi, I would like to define the following parametric function type struct myParamFun fun::Function param end (pf::myParamFun)(x...)=pf.fun(x...,pf.param) which works fine. However, when broadcasting over C…

---

## [New to Julia and Formatting StructArrays for GPU use with mutable scalars](https://discourse.julialang.org/t/new-to-julia-and-formatting-structarrays-for-gpu-use-with-mutable-scalars/94243)

<div class="topic-metadata">

**Author:** [@David\_A](https://discourse.julialang.org/u/David_A)\
**Replies:** 2\
**Last updated:** [February 13, 2023, 11:30pm UTC](https://discourse.julialang.org/t/new-to-julia-and-formatting-structarrays-for-gpu-use-with-mutable-scalars/94243 "2023-02-13T23:30:46Z")

</div>

First, I apologize if the long rambling post is not in keeping with expectations here and would like to preemptively thank anyone with the patience to read part or all of this and provide any feedback/guidance. Backgrou…

---

## [Question about CUDA kernels](https://discourse.julialang.org/t/question-about-cuda-kernels/94345)

<div class="topic-metadata">

**Author:** [@EduMolinero](https://discourse.julialang.org/u/EduMolinero)\
**Replies:** 4\
**Last updated:** [February 10, 2023, 3:33pm UTC](https://discourse.julialang.org/t/question-about-cuda-kernels/94345 "2023-02-10T15:33:02Z")

</div>

Hi! I’m struggling to implement the following CUDA kernel: using CUDA using LinearAlgebra function kernel!(A, B, C) X = CUDA.size(A)\[3\] Y = CUDA.size(B)\[3\] for j in 1:X for i in 1:Y @in…

---

## [Using CuSparseMatrixCSR](https://discourse.julialang.org/t/using-cusparsematrixcsr/94379)

<div class="topic-metadata">

**Author:** [@shakedregev](https://discourse.julialang.org/u/shakedregev)\
**Replies:** 1\
**Last updated:** [February 9, 2023, 11:09pm UTC](https://discourse.julialang.org/t/using-cusparsematrixcsr/94379 "2023-02-09T23:09:35Z")

</div>

Hi, Based on the CUDA.jl github I tried to create a CuSparseMatrixCSR matrix. I’m sparing the loading of the data, all of the values I input in are defined. using CUDA export CuSparseMatrixCSC, CuSparseMatrixCSR, CuSp…

---

## [How to resize and fill more efficiently?](https://discourse.julialang.org/t/how-to-resize-and-fill-more-efficiently/94081)

<div class="topic-metadata">

**Author:** [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Replies:** 14\
**Last updated:** [February 9, 2023, 7:16pm UTC](https://discourse.julialang.org/t/how-to-resize-and-fill-more-efficiently/94081 "2023-02-09T19:16:37Z")

</div>

Hello! Suppose I have: using StaticArrays S = rand(SVector{3,Float32},10^6) function MyResizeAndFill!(S,N) resize!(S,N) fill!(S,zero(eltype(S))) end @benchmark MyResizeAndFill!($S,$(10^8)) BenchmarkTools.Tri…

---

## [Why can Flux not reduce this?](https://discourse.julialang.org/t/why-can-flux-not-reduce-this/94071)

<div class="topic-metadata">

**Author:** [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Replies:** 5\
**Last updated:** [February 8, 2023, 7:57pm UTC](https://discourse.julialang.org/t/why-can-flux-not-reduce-this/94071 "2023-02-08T19:57:55Z")

</div>

Hello I am trying: using CUDA S = rand(SVector{3,Float32},5) DST = zeros(SVector{3,Float32},5) I = \[3,1,2,5,4\] # Works great on CPU @CUDA.time NNlib.scatter!(+, DST,S,I) 0.000004 seconds 5-element Vecto…

---

## [GPU code has a high amount of CPU allocations?](https://discourse.julialang.org/t/gpu-code-has-a-high-amount-of-cpu-allocations/94092)

<div class="topic-metadata">

**Author:** [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Replies:** 7\
**Last updated:** [February 8, 2023, 2:17pm UTC](https://discourse.julialang.org/t/gpu-code-has-a-high-amount-of-cpu-allocations/94092 "2023-02-08T14:17:50Z")

</div>

Hello! More of a “meta” question. I need some pointers on this. I have some Julia code which allocates 0 times on the GPU other than the initial making of arrays for memory etc. But I have such a high number of CPU al…

---

## [CUDNN produces a lot of errors and warnings](https://discourse.julialang.org/t/cudnn-produces-a-lot-of-errors-and-warnings/93940)

<div class="topic-metadata">

**Author:** [@Euhan](https://discourse.julialang.org/u/Euhan)\
**Replies:** 2\
**Last updated:** [February 8, 2023, 9:14am UTC](https://discourse.julialang.org/t/cudnn-produces-a-lot-of-errors-and-warnings/93940 "2023-02-08T09:14:54Z")

</div>

I have lately been getting a lot of errors emanating from CUDNN.jl when training Flux models. This happens also with code I really think has or would have worked before. Does anyone understand this? ┌ Error: CuDNN (v83…

---

## [Passing custom struct into kernel function](https://discourse.julialang.org/t/passing-custom-struct-into-kernel-function/94069)

<div class="topic-metadata">

**Author:** [@FujiwaraTakumiEH](https://discourse.julialang.org/u/FujiwaraTakumiEH)\
**Replies:** 1\
**Last updated:** [February 4, 2023, 10:43pm UTC](https://discourse.julialang.org/t/passing-custom-struct-into-kernel-function/94069 "2023-02-04T22:43:19Z")

</div>

Hello, I’m still a little confused about Adapt.jl. I read several posts in the forum and tried it for my case, but it did not work…:joy: I have a custom struct like this: using Adapt using CUDA # T is either Float64 …

---

## [How to create sliced views of \`CuArray\` correctly?](https://discourse.julialang.org/t/how-to-create-sliced-views-of-cuarray-correctly/93971)

<div class="topic-metadata">

**Author:** [@jinfreedom](https://discourse.julialang.org/u/jinfreedom)\
**Replies:** 1\
**Last updated:** [February 3, 2023, 9:38pm UTC](https://discourse.julialang.org/t/how-to-create-sliced-views-of-cuarray-correctly/93971 "2023-02-03T21:38:45Z")

</div>

Hi, I would like to split and reshape a CuArray to a StructArray “in place” as views. I used sliced views of the input CuArray and got a warning about slicing. Is there a correct way of doing the following? julia\> siz=(…

---

## [LoadError: @atomic modify expression missing field access](https://discourse.julialang.org/t/loaderror-atomic-modify-expression-missing-field-access/93973)

<div class="topic-metadata">

**Author:** [@bcsj](https://discourse.julialang.org/u/bcsj)\
**Replies:** 1\
**Last updated:** [February 3, 2023, 8:18am UTC](https://discourse.julialang.org/t/loaderror-atomic-modify-expression-missing-field-access/93973 "2023-02-03T08:18:35Z")

</div>

I’ve been trying to write a reduction kernel using \[1\] as a blueprint to work from, but I kept getting this error message: LoadError: @atomic modify expression missing field access I couldn’t understand why, and in t…

---

## [How to improve the calculation speed of this kernel on GPU?](https://discourse.julialang.org/t/how-to-improve-the-calculation-speed-of-this-kernel-on-gpu/93831)

<div class="topic-metadata">

**Author:** [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Replies:** 2\
**Last updated:** [February 2, 2023, 1:10pm UTC](https://discourse.julialang.org/t/how-to-improve-the-calculation-speed-of-this-kernel-on-gpu/93831 "2023-02-02T13:10:26Z")

</div>

Hello everyone! I have worked quite a bit on setting up a robust foundation for structuring my code and computation loop, with quite a few good pointers from you guys on the forum, so I hope to be able to learn more fro…

---

## [Batched Matrix solve in CUDA.jl](https://discourse.julialang.org/t/batched-matrix-solve-in-cuda-jl/52855)

<div class="topic-metadata">

**Author:** [@jpdoane](https://discourse.julialang.org/u/jpdoane)\
**Replies:** 3\
**Last updated:** [February 1, 2023, 7:00pm UTC](https://discourse.julialang.org/t/batched-matrix-solve-in-cuda-jl/52855 "2023-02-01T19:00:38Z")

</div>

Is there a way to perform batched matrix inversion/ldiv in CUDA.jl, ie: Yi = Ri \\ Xi where Ri is NxN, and Yi and Xi are NxM, with N and M small (~12). I need to solve a large number of these systems, e.g. 1 \< i \< 50k.…

---

## [Could CUDA.jl be better at explaining what went wrong](https://discourse.julialang.org/t/could-cuda-jl-be-better-at-explaining-what-went-wrong/93735)

<div class="topic-metadata">

**Author:** [@Euhan](https://discourse.julialang.org/u/Euhan)\
**Replies:** 8\
**Last updated:** [January 30, 2023, 2:51pm UTC](https://discourse.julialang.org/t/could-cuda-jl-be-better-at-explaining-what-went-wrong/93735 "2023-01-30T14:51:37Z")

</div>

Wouldn’t it be possible to include a bit more information in the CUDA error messages? It is often hard to locate the source of an error that manifests itself in the CUDA code and I would think all clues would be welcome…

---

## [Working with shared memory as one or more variables, what is a good approach?](https://discourse.julialang.org/t/working-with-shared-memory-as-one-or-more-variables-what-is-a-good-approach/93758)

<div class="topic-metadata">

**Author:** [@bcsj](https://discourse.julialang.org/u/bcsj)\
**Replies:** 2\
**Last updated:** [January 30, 2023, 1:14pm UTC](https://discourse.julialang.org/t/working-with-shared-memory-as-one-or-more-variables-what-is-a-good-approach/93758 "2023-01-30T13:14:29Z")

</div>

I’m somewhat new to writing gpu code, but I am working on a somewhat complicated kernel function, which involves a number of steps in which shared memory is used to speed up the computations. To be efficient I reuse the…

---

## [How to properly pass structs into GPU? (MWE included)](https://discourse.julialang.org/t/how-to-properly-pass-structs-into-gpu-mwe-included/93727)

<div class="topic-metadata">

**Author:** [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Replies:** 6\
**Last updated:** [January 29, 2023, 8:23pm UTC](https://discourse.julialang.org/t/how-to-properly-pass-structs-into-gpu-mwe-included/93727 "2023-01-29T20:23:50Z")

</div>

Hello! Lately I’ve been asking a few questions about how to program on GPU etc. and I’ve gotten some answers (thank you very much!), but of course it is hard to help me if I only provide a few line of codes. So this mor…

---

## [Passing mutable struct to kernel](https://discourse.julialang.org/t/passing-mutable-struct-to-kernel/71948)

<div class="topic-metadata">

**Author:** [@lodygaw](https://discourse.julialang.org/u/lodygaw)\
**Replies:** 7\
**Last updated:** [January 29, 2023, 3:20pm UTC](https://discourse.julialang.org/t/passing-mutable-struct-to-kernel/71948 "2023-01-29T15:20:27Z")

</div>

Hi, I am looking for the best way to use structs inside the kernels and I am having hard time with understanding whole mutable/immutable/isbits/inline thing. My goal is to be able to pass struct as an argument to a kern…

---

## [How do I get allocations down but keep code speed?](https://discourse.julialang.org/t/how-do-i-get-allocations-down-but-keep-code-speed/93692)

<div class="topic-metadata">

**Author:** [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Replies:** 9\
**Last updated:** [January 28, 2023, 9:11pm UTC](https://discourse.julialang.org/t/how-do-i-get-allocations-down-but-keep-code-speed/93692 "2023-01-28T21:11:17Z")

</div>

using CUDA, StaticArrays, Flux N = 10^6 T = Float32 WgL = CuArray(rand(SVector{3,T},N)) I\_ = CuArray(rand(1:6195,N)) J\_ = CuArray(rand(1:6195,N)) WGL = CuArray(zeros(SVector{3,T},N)) CUDA.@allocated WGL .= WgL\[I\_\] -…

---

## [Why does GPU allocate so much for me?](https://discourse.julialang.org/t/why-does-gpu-allocate-so-much-for-me/93681)

<div class="topic-metadata">

**Author:** [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Replies:** 1\
**Last updated:** [January 28, 2023, 9:41am UTC](https://discourse.julialang.org/t/why-does-gpu-allocate-so-much-for-me/93681 "2023-01-28T09:41:35Z")

</div>

Hello! Probably a noob question, but got to ask :slight\_smile: Suppose I have: using CUDA N = 10^6 AGPU = CuArray(rand(N)) BGPU = CuArray(rand(N)) ACPU = rand(N) BCPU = rand(N) # Why does GPU allocate so much mo…

---

## [Unable to use CUDA from artifacts](https://discourse.julialang.org/t/unable-to-use-cuda-from-artifacts/86854)

<div class="topic-metadata">

**Author:** [@yoshi](https://discourse.julialang.org/u/yoshi)\
**Replies:** 5\
**Last updated:** [January 28, 2023, 8:06am UTC](https://discourse.julialang.org/t/unable-to-use-cuda-from-artifacts/86854 "2023-01-28T08:06:46Z")

</div>

I’m trying to use julia in an HPC environment. I load the CUDA driver using slurm’s “module” functionality. I can see where it is in the $PATH variable. But when I go to run “pkg\> test CUDA” I get warnings saying “This v…

[Previous page](https://discourse.julialang.org/c/domain/gpu/11.md?page=12)

[Next page](https://discourse.julialang.org/c/domain/gpu/11.md?page=14)