# GPU

**URL:** https://discourse.julialang.org/c/domain/gpu/11.md?page=6

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 7

---

## [Synchronize streams in CUDA.jl](https://discourse.julialang.org/t/synchronize-streams-in-cuda-jl/118127)

<div class="topic-metadata">

**Author:** [@Cibin\_Joseph](https://discourse.julialang.org/u/Cibin_Joseph)\
**Replies:** 11\
**Last updated:** [August 23, 2024, 10:58pm UTC](https://discourse.julialang.org/t/synchronize-streams-in-cuda-jl/118127 "2024-08-23T22:58:47Z")

</div>

Which of the synchronize commands is supposed to be used to synchronize the streams in CUDA? I am able to synchronize them with device\_synchronize(), but this seems to have a huge impact on speed. Is there perhaps a more…

---

## [Most efficient way to find cholesky decomposition of slices of a 3D array in KernelAbstractions](https://discourse.julialang.org/t/most-efficient-way-to-find-cholesky-decomposition-of-slices-of-a-3d-array-in-kernelabstractions/118292)

<div class="topic-metadata">

**Author:** [@Jarrod-Angove](https://discourse.julialang.org/u/Jarrod-Angove)\
**Replies:** 4\
**Last updated:** [August 21, 2024, 11:01pm UTC](https://discourse.julialang.org/t/most-efficient-way-to-find-cholesky-decomposition-of-slices-of-a-3d-array-in-kernelabstractions/118292 "2024-08-21T23:01:30Z")

</div>

I am using KernelAbstractions.jl to write a machine learning package that is device-agnostic. At one point in my algorithm, I end up with a NxNxn tensor, let’s call it T, on the device. I need to create a new vector, le…

---

## [Putting obj files on the GPU with Metal.jl](https://discourse.julialang.org/t/putting-obj-files-on-the-gpu-with-metal-jl/118416)

<div class="topic-metadata">

**Author:** [@dchang10](https://discourse.julialang.org/u/dchang10)\
**Replies:** 0\
**Last updated:** [August 20, 2024, 1:58pm UTC](https://discourse.julialang.org/t/putting-obj-files-on-the-gpu-with-metal-jl/118416 "2024-08-20T13:58:54Z")

</div>

How do I bring mesh from obj files onto the GPU? I am trying to do some GPU manipulation of meshes from .obj files. I was testing my code with the stanford bunny obj file, which I imported into a GeometryBasics Mesh usi…

---

## [Invalid LLVM IR error using CUDA](https://discourse.julialang.org/t/invalid-llvm-ir-error-using-cuda/118244)

<div class="topic-metadata">

**Author:** [@0samuraiE](https://discourse.julialang.org/u/0samuraiE)\
**Replies:** 3\
**Last updated:** [August 17, 2024, 5:26am UTC](https://discourse.julialang.org/t/invalid-llvm-ir-error-using-cuda/118244 "2024-08-17T05:26:18Z")

</div>

Hi all. I have encountered a strange problem and would like to ask you a question. While cpu function works, gpu function fails. Why? Can I do such override in CUDA kernel? It seems that cuda is confusing Int32 and In…

---

## [Raspberry Pi AI Kit](https://discourse.julialang.org/t/raspberry-pi-ai-kit/118225)

<div class="topic-metadata">

**Author:** [@Vyacheslav\_Vorobyov](https://discourse.julialang.org/u/Vyacheslav_Vorobyov)\
**Replies:** 1\
**Last updated:** [August 15, 2024, 6:25pm UTC](https://discourse.julialang.org/t/raspberry-pi-ai-kit/118225 "2024-08-15T18:25:49Z")

</div>

Hi all! Does Julia support Raspberry Pi AI Kit ? Or is it possible to do that in some hard (not very) way? Raspberry Pi AI Kit – is a Raspberry Pi with Hailo AI acceleration module on top. The context is that I was …

---

## [Different results with different macros for profiling](https://discourse.julialang.org/t/different-results-with-different-macros-for-profiling/117592)

<div class="topic-metadata">

**Author:** [@HAP40HTT](https://discourse.julialang.org/u/HAP40HTT)\
**Replies:** 3\
**Last updated:** [August 6, 2024, 7:53pm UTC](https://discourse.julialang.org/t/different-results-with-different-macros-for-profiling/117592 "2024-08-06T19:53:09Z")

</div>

Hi everyone, I’m relatively new to kernel programming with CUDA.jl and am currently implementing a kernel to compute attention using a sliding-window attention mechanism. While profiling the code, I observed that the r…

---

## [Using views of CuArray with CUDA-aware MPI is extremely slow](https://discourse.julialang.org/t/using-views-of-cuarray-with-cuda-aware-mpi-is-extremely-slow/117348)

<div class="topic-metadata">

**Author:** [@0samuraiE](https://discourse.julialang.org/u/0samuraiE)\
**Replies:** 14\
**Last updated:** [August 5, 2024, 12:36pm UTC](https://discourse.julialang.org/t/using-views-of-cuarray-with-cuda-aware-mpi-is-extremely-slow/117348 "2024-08-05T12:36:56Z")

</div>

Hi everyone, I attempted a simple 2D boundary communication using MPI.jl and CUDA.jl. However, when I used views, the wall time was 100 times slower compared to not using views. I made slight modifications based on the …

---

## [Delays shown in Nsight Systems between HtoD memcopy and kernel launch when using CUDA.jl](https://discourse.julialang.org/t/delays-shown-in-nsight-systems-between-htod-memcopy-and-kernel-launch-when-using-cuda-jl/117446)

<div class="topic-metadata">

**Author:** [@Cibin\_Joseph](https://discourse.julialang.org/u/Cibin_Joseph)\
**Replies:** 9\
**Last updated:** [July 31, 2024, 11:22pm UTC](https://discourse.julialang.org/t/delays-shown-in-nsight-systems-between-htod-memcopy-and-kernel-launch-when-using-cuda-jl/117446 "2024-07-31T23:22:04Z")

</div>

For the CUDA.jl minimal example shown below, I see a considerably large delay between the HtoD memcopy and the kernel launch. I’m not able to figure out what causes it. For a larger array, sometimes this delay is much le…

---

## [Test a package with a CUDA dependency through GitHub Actions CI](https://discourse.julialang.org/t/test-a-package-with-a-cuda-dependency-through-github-actions-ci/117702)

<div class="topic-metadata">

**Author:** [@huiyuxie](https://discourse.julialang.org/u/huiyuxie)\
**Replies:** 2\
**Last updated:** [July 31, 2024, 8:42pm UTC](https://discourse.julialang.org/t/test-a-package-with-a-cuda-dependency-through-github-actions-ci/117702 "2024-07-31T20:42:13Z")

</div>

Hi all! I am developing a new package integrating CUDA.jl as a dependency. However, I encountered errors indicating CUDA driver not found in CI when testing. I am using Standard GitHub-hosted runners and do not want to s…

---

## [Dynamic parallelism slow in CUDA.jl](https://discourse.julialang.org/t/dynamic-parallelism-slow-in-cuda-jl/117463)

<div class="topic-metadata">

**Author:** [@z-wang](https://discourse.julialang.org/u/z-wang)\
**Replies:** 1\
**Last updated:** [July 25, 2024, 6:09pm UTC](https://discourse.julialang.org/t/dynamic-parallelism-slow-in-cuda-jl/117463 "2024-07-25T18:09:50Z")

</div>

Hi, i’m learning dynamic parallelism in CUDA.jl . Here i have a code which executes a parent kernel concurrently 1000 times, and each parent kernel queues the child kernel N times. My problem is that this code is pretty …

---

## [CUDA: CUDA driver not found](https://discourse.julialang.org/t/cuda-cuda-driver-not-found/117186)

<div class="topic-metadata">

**Author:** [@engram](https://discourse.julialang.org/u/engram)\
**Replies:** 24\
**Last updated:** [July 24, 2024, 5:08am UTC](https://discourse.julialang.org/t/cuda-cuda-driver-not-found/117186 "2024-07-24T05:08:21Z")

</div>

A few days ago the IT-department of my company added a virtual GPU to my virtual machine which I would like to use in Julia. I use julia 1.10.3 (which I can load as a module through (module load julia/1.10.3) and have a…

---

## [Variable scoping issue when using multiple GPUs in CUDA.jl](https://discourse.julialang.org/t/variable-scoping-issue-when-using-multiple-gpus-in-cuda-jl/117152)

<div class="topic-metadata">

**Author:** [@Cibin\_Joseph](https://discourse.julialang.org/u/Cibin_Joseph)\
**Replies:** 1\
**Last updated:** [July 17, 2024, 8:27pm UTC](https://discourse.julialang.org/t/variable-scoping-issue-when-using-multiple-gpus-in-cuda-jl/117152 "2024-07-17T20:27:12Z")

</div>

I’m using two for loops: one, to iterate over CUDA devices and launch kernels. The second, to iterate over CUDA devices and copy results back to the CPU. When I do this, I get an ‘undefined variable’ error due to local s…

---

## [Suggestion: abstraction for integrated GPUs?](https://discourse.julialang.org/t/suggestion-abstraction-for-integrated-gpus/117061)

<div class="topic-metadata">

**Author:** [@bremez](https://discourse.julialang.org/u/bremez)\
**Replies:** 7\
**Last updated:** [July 16, 2024, 7:06am UTC](https://discourse.julialang.org/t/suggestion-abstraction-for-integrated-gpus/117061 "2024-07-16T07:06:16Z")

</div>

I have had the following thought: many people do not have regular access to HPC or high-end GPUs (where Nvidia seems most popular), and/or prioritize the benefit of interactive sessions on their personal laptops and low …

---

## [Metal Kernel 3D indices](https://discourse.julialang.org/t/metal-kernel-3d-indices/116429)

<div class="topic-metadata">

**Author:** [@trasor](https://discourse.julialang.org/u/trasor)\
**Replies:** 4\
**Last updated:** [July 14, 2024, 6:47pm UTC](https://discourse.julialang.org/t/metal-kernel-3d-indices/116429 "2024-07-14T18:47:19Z")

</div>

Hello together, i try to write a metal kernel that do some operations on 3D arrays like: function vadd!( nz::Int64,ny::Int64,nx::Int64, a::MtlDeviceArray{Float32, 3, 1}, b::MtlDeviceArray{Float32, 3, 1}, c::MtlDeviceArr…

---

## [External functions in GPU ODE example](https://discourse.julialang.org/t/external-functions-in-gpu-ode-example/116719)

<div class="topic-metadata">

**Author:** [@xztt](https://discourse.julialang.org/u/xztt)\
**Replies:** 7\
**Last updated:** [July 11, 2024, 9:07pm UTC](https://discourse.julialang.org/t/external-functions-in-gpu-ode-example/116719 "2024-07-11T21:07:36Z")

</div>

Hi all, I have tried adapting the DiffEqGPU example to my own example which depends on some external functions. The error is quite dense so if someone could point me in the right direction this would be a big help. Exa…

---

## [Passing \`::Type{InconcreteType}\` to CUDA kernel](https://discourse.julialang.org/t/passing-type-inconcretetype-to-cuda-kernel/116833)

<div class="topic-metadata">

**Author:** [@hexaeder](https://discourse.julialang.org/u/hexaeder)\
**Replies:** 2\
**Last updated:** [July 11, 2024, 12:04pm UTC](https://discourse.julialang.org/t/passing-type-inconcretetype-to-cuda-kernel/116833 "2024-07-11T12:04:32Z")

</div>

Hello, I have some kernels that share a lot of code but specialize in the end based on some type of AbstractAction in the MWE below Addition or Multiplication. Those concrete actions have some nonbitstype metadata, whic…

---

## [CUDA.compute\_sanitizer() method not defined](https://discourse.julialang.org/t/cuda-compute-sanitizer-method-not-defined/116888)

<div class="topic-metadata">

**Author:** [@alexpan5](https://discourse.julialang.org/u/alexpan5)\
**Replies:** 1\
**Last updated:** [July 11, 2024, 10:18am UTC](https://discourse.julialang.org/t/cuda-compute-sanitizer-method-not-defined/116888 "2024-07-11T10:18:31Z")

</div>

I want to use compute-sanitizer to look into a memory error, but when I run: julia\> using CUDA julia\> CUDA.compute\_sanitizer() I just get ERROR: UndefVarError: compute\_sanitizer not defined. CUDA.compute\_sanitizer\_cmd…

---

## [Rubbish values from gpu kernel](https://discourse.julialang.org/t/rubbish-values-from-gpu-kernel/116632)

<div class="topic-metadata">

**Author:** [@Abdelrahman912](https://discourse.julialang.org/u/Abdelrahman912)\
**Replies:** 5\
**Last updated:** [July 8, 2024, 3:06pm UTC](https://discourse.julialang.org/t/rubbish-values-from-gpu-kernel/116632 "2024-07-08T15:06:46Z")

</div>

Hi, I am trying to assemble global stiffness matrix using GPU and coloring algorithm, so after coloring (i.e. group non adjacent cells in the same color) I have to launch a kernel for each color and by doing so, we don’…

---

## [Basic NN forward passage: CuArray fine, oneAPI array:](https://discourse.julialang.org/t/basic-nn-forward-passage-cuarray-fine-oneapi-array/116351)

<div class="topic-metadata">

**Author:** [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Replies:** 0\
**Last updated:** [June 28, 2024, 10:21am UTC](https://discourse.julialang.org/t/basic-nn-forward-passage-cuarray-fine-oneapi-array/116351 "2024-06-28T10:21:25Z")

</div>

Hello, I am “testing” GPU programming with a mock forward passage of a basic NN: using Test, oneAPI, BenchmarkTools, LinearAlgebra relu(x) = max(0,x) forward\_layer(x,w,w0,f) = f.(w\*x .+ w0) function forward\_network!…

---

## [Do a function like relu need a kernel ? When you need to write a GPU kernel rather than "just" using CuArray?](https://discourse.julialang.org/t/do-a-function-like-relu-need-a-kernel-when-you-need-to-write-a-gpu-kernel-rather-than-just-using-cuarray/116265)

<div class="topic-metadata">

**Author:** [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Replies:** 3\
**Last updated:** [June 26, 2024, 4:54pm UTC](https://discourse.julialang.org/t/do-a-function-like-relu-need-a-kernel-when-you-need-to-write-a-gpu-kernel-rather-than-just-using-cuarray/116265 "2024-06-26T16:54:03Z")

</div>

The relu function works elementwise by returning the input that is non-negative, i.e. relu(x) = ifelse.(x .\> 0, x, 0). Does the fact that it works elementwise means I need to write a GPU kernel for it, or I can simply u…

---

## [CUDA: unsupported dynamic function invocation for closure](https://discourse.julialang.org/t/cuda-unsupported-dynamic-function-invocation-for-closure/115981)

<div class="topic-metadata">

**Author:** [@wsshin](https://discourse.julialang.org/u/wsshin)\
**Replies:** 3\
**Last updated:** [June 23, 2024, 12:07am UTC](https://discourse.julialang.org/t/cuda-unsupported-dynamic-function-invocation-for-closure/115981 "2024-06-23T00:07:37Z")

</div>

I have a function f that takes a scalar parameter and a vector. It works well for CuVector: xs = CuVector{Float64}(rand(10)) f.(param, xs) # works well However, if I create a closure out of f such that it holds param,…

---

## [Running CUDA.jl with JULIA\_DEPOT\_PATH read-only](https://discourse.julialang.org/t/running-cuda-jl-with-julia-depot-path-read-only/115833)

<div class="topic-metadata">

**Author:** [@erin-atkinson](https://discourse.julialang.org/u/erin-atkinson)\
**Replies:** 8\
**Last updated:** [June 21, 2024, 7:55am UTC](https://discourse.julialang.org/t/running-cuda-jl-with-julia-depot-path-read-only/115833 "2024-06-21T07:55:22Z")

</div>

I’m updating from Julia 1.8 to 1.10 for the simulations I’m running. Previously everything worked fine, but now I have an issue. I can install CUDA.jl just fine and use it on the HPC login nodes, but on compute nodes the…

---

## [Enzyme Cuda dynamic memory](https://discourse.julialang.org/t/enzyme-cuda-dynamic-memory/111063)

<div class="topic-metadata">

**Author:** [@Jakub\_Mitura](https://discourse.julialang.org/u/Jakub_Mitura)\
**Replies:** 12\
**Last updated:** [June 17, 2024, 3:56pm UTC](https://discourse.julialang.org/t/enzyme-cuda-dynamic-memory/111063 "2024-06-17T15:56:18Z")

</div>

Hello I am trying to use enzyme with dynamic shared memory but I get error that probably is related to the fact that I am trying to instantiate dynamic memory in not device code. However If not use it like that how to ma…

---

## [Passing a structure with constructor on GPU](https://discourse.julialang.org/t/passing-a-structure-with-constructor-on-gpu/115596)

<div class="topic-metadata">

**Author:** [@Baptiste\_Fabre](https://discourse.julialang.org/u/Baptiste_Fabre)\
**Replies:** 2\
**Last updated:** [June 17, 2024, 1:06pm UTC](https://discourse.julialang.org/t/passing-a-structure-with-constructor-on-gpu/115596 "2024-06-17T13:06:32Z")

</div>

Dear all, I have a question about how to pass a structure built with a constructor on a GPU card. My structure is defined as struct FieldParams{T} I₀ :: T λ₀ :: T α :: T r :: T ω :: T E₀ :: T Up :: T T₀ :: T N…

---

## [CUB wrapper ccall overhead on Windows](https://discourse.julialang.org/t/cub-wrapper-ccall-overhead-on-windows/115585)

<div class="topic-metadata">

**Author:** [@eldee](https://discourse.julialang.org/u/eldee)\
**Replies:** 10\
**Last updated:** [June 15, 2024, 10:37am UTC](https://discourse.julialang.org/t/cub-wrapper-ccall-overhead-on-windows/115585 "2024-06-15T10:37:35Z")

</div>

Hi, To get a faster GPU sort + sortperm for a specific use case, I succesfully wrapped CUB’s SortPairs method for sorting 64-bit integer keys (sort) and 32-bit integer values (sortperm). However, when measuring the exec…

---

## [Calling repeat with a CUDA array changes the state of the random number generator](https://discourse.julialang.org/t/calling-repeat-with-a-cuda-array-changes-the-state-of-the-random-number-generator/115598)

<div class="topic-metadata">

**Author:** [@Alexander-Barth](https://discourse.julialang.org/u/Alexander-Barth)\
**Replies:** 2\
**Last updated:** [June 14, 2024, 6:09am UTC](https://discourse.julialang.org/t/calling-repeat-with-a-cuda-array-changes-the-state-of-the-random-number-generator/115598 "2024-06-14T06:09:30Z")

</div>

I am trying to refactor a code with uses random numbers and CUDA arrays. Quite surprisingly the function repeat changes the sequence of random numbers. I am wondering if other people see the same behavior. Do I miss s…

---

## [Difference between GPUArrays.jl and KernelAbstractations.jl](https://discourse.julialang.org/t/difference-between-gpuarrays-jl-and-kernelabstractations-jl/115457)

<div class="topic-metadata">

**Author:** [@albertomercurio](https://discourse.julialang.org/u/albertomercurio)\
**Replies:** 4\
**Last updated:** [June 12, 2024, 2:17pm UTC](https://discourse.julialang.org/t/difference-between-gpuarrays-jl-and-kernelabstractations-jl/115457 "2024-06-12T14:17:50Z")

</div>

Hello, I would like to implement some simple matrix-vector multiplication, when none of them are strided arrays (e.g., @view(A\[\[1, 4, 5\], 1:10\])). Standard LinearAlgebra.mul! doesn’t support these arrays, so I thought t…

---

## [Scalar Indexing error when performing mul!(A, B, C) with A a view of a Matrix](https://discourse.julialang.org/t/scalar-indexing-error-when-performing-mul-a-b-c-with-a-a-view-of-a-matrix/115439)

<div class="topic-metadata">

**Author:** [@albertomercurio](https://discourse.julialang.org/u/albertomercurio)\
**Replies:** 3\
**Last updated:** [June 11, 2024, 4:49pm UTC](https://discourse.julialang.org/t/scalar-indexing-error-when-performing-mul-a-b-c-with-a-a-view-of-a-matrix/115439 "2024-06-11T16:49:53Z")

</div>

Hello, I paste also here the issue raised here. Describe the bug The in-place multiplication mul!(A, B, C) fails when A is a vector view of a matrix. To reproduce The Minimal Working Example (MWE) for this bug: us…

---

## [Slicing a CuArray in a kernel](https://discourse.julialang.org/t/slicing-a-cuarray-in-a-kernel/115285)

<div class="topic-metadata">

**Author:** [@peteole](https://discourse.julialang.org/u/peteole)\
**Replies:** 1\
**Last updated:** [June 6, 2024, 4:21pm UTC](https://discourse.julialang.org/t/slicing-a-cuarray-in-a-kernel/115285 "2024-06-06T16:21:35Z")

</div>

Hi, I would like to take a slice from a CuArray in a kernel: using CUDA using StaticArrays using LinearAlgebra function run() a::Float64=5.0 n=3 vectors = CUDA.rand(Float64, 100,n) #vectors=CuArray(\[@S…

---

## [Simple CUDA kernel on matrix slower than running GPU](https://discourse.julialang.org/t/simple-cuda-kernel-on-matrix-slower-than-running-gpu/115018)

<div class="topic-metadata">

**Author:** [@jlbosse](https://discourse.julialang.org/u/jlbosse)\
**Replies:** 8\
**Last updated:** [June 3, 2024, 12:27pm UTC](https://discourse.julialang.org/t/simple-cuda-kernel-on-matrix-slower-than-running-gpu/115018 "2024-06-03T12:27:21Z")

</div>

I am trying to write a simple CUDA kernel that swaps pairs of columns in a square matrix and puts a sign on one of them: using CUDA, BenchmarkTools function swap\_and\_sign!(A::Matrix) n = size(A, 1) ÷ 2 for i in…

[Previous page](https://discourse.julialang.org/c/domain/gpu/11.md?page=5)

[Next page](https://discourse.julialang.org/c/domain/gpu/11.md?page=7)