# GPU

**URL:** https://discourse.julialang.org/c/domain/gpu/11.md?page=12

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 13

---

## [Random variations between results of CPU and GPU computation](https://discourse.julialang.org/t/random-variations-between-results-of-cpu-and-gpu-computation/98517)

<div class="topic-metadata">

**Author:** [@ReWoP](https://discourse.julialang.org/u/ReWoP)\
**Replies:** 7\
**Last updated:** [May 9, 2023, 7:02pm UTC](https://discourse.julialang.org/t/random-variations-between-results-of-cpu-and-gpu-computation/98517 "2023-05-09T19:02:20Z")

</div>

Hi all, I’ve noticed that I’m getting some variations between the results computed in the GPU VS the ones obtained in the CPU. Here is a simple example: a = CUDA.rand(Float64, 10000); @show CUDA.sum(a) - sum(Array(a)…

---

## [Why does the execution time of overlapping GPU and CPU computations not get faster after using the Mem.pin() function?](https://discourse.julialang.org/t/why-does-the-execution-time-of-overlapping-gpu-and-cpu-computations-not-get-faster-after-using-the-mem-pin-function/98257)

<div class="topic-metadata">

**Author:** [@fred\_wu](https://discourse.julialang.org/u/fred_wu)\
**Replies:** 3\
**Last updated:** [May 5, 2023, 9:24am UTC](https://discourse.julialang.org/t/why-does-the-execution-time-of-overlapping-gpu-and-cpu-computations-not-get-faster-after-using-the-mem-pin-function/98257 "2023-05-05T09:24:53Z")

</div>

Hi, I have a question about using Mem.pin() function in GPU concurrent programming. According to my understanding, the GPU and CPU will block when copying data and cannot perform asynchronous operations. However, CPU bl…

---

## [Custom Flux layer looking weird upon profiling](https://discourse.julialang.org/t/custom-flux-layer-looking-weird-upon-profiling/98273)

<div class="topic-metadata">

**Author:** [@AMart](https://discourse.julialang.org/u/AMart)\
**Replies:** 1\
**Last updated:** [May 3, 2023, 8:11pm UTC](https://discourse.julialang.org/t/custom-flux-layer-looking-weird-upon-profiling/98273 "2023-05-03T20:11:21Z")

</div>

I’m developing some custom Conv and Dense layers to use with Flux for my research and I’m having trouble understanding the profiler output. I’m using LeNet from Flux examples to study my implementation. The following co…

---

## [Render Pipeline in Metal.jl](https://discourse.julialang.org/t/render-pipeline-in-metal-jl/97847)

<div class="topic-metadata">

**Author:** [@dchang10](https://discourse.julialang.org/u/dchang10)\
**Replies:** 9\
**Last updated:** [April 30, 2023, 5:07pm UTC](https://discourse.julialang.org/t/render-pipeline-in-metal-jl/97847 "2023-04-30T17:07:49Z")

</div>

I’m currently trying to set up a render pipeline with Metal.jl by defining a vertex shader, rasterizer and fragment shader, but am not sure how to go about it. Is there an example or description of how this is done with …

---

## [multiple-GPUs per process](https://discourse.julialang.org/t/multiple-gpus-per-process/98008)

<div class="topic-metadata">

**Author:** [@bjarthur](https://discourse.julialang.org/u/bjarthur)\
**Replies:** 3\
**Last updated:** [April 27, 2023, 8:18pm UTC](https://discourse.julialang.org/t/multiple-gpus-per-process/98008 "2023-04-27T20:18:26Z")

</div>

i’m not sure whether i should file an issue on github because this is a bug, or post a plea for help here because i’m doing something wrong, but riffing on the multiple GPUs per process example, and adding to it an Array…

---

## [KernelAbstractions.get\_backend keyword arguments](https://discourse.julialang.org/t/kernelabstractions-get-backend-keyword-arguments/97895)

<div class="topic-metadata">

**Author:** [@bjarthur](https://discourse.julialang.org/u/bjarthur)\
**Replies:** 1\
**Last updated:** [April 26, 2023, 2:36am UTC](https://discourse.julialang.org/t/kernelabstractions-get-backend-keyword-arguments/97895 "2023-04-26T02:36:23Z")

</div>

CUDABackend accepts two keyword arguments. the default for prefer\_blocks is false. i have a custom kernel for which i need that to be true. KernelAbstractions.get\_backend has no way to pass that in. so how do i write …

---

## [Question about coalesced read and write to the global memory using CUDA.jl 2D grid](https://discourse.julialang.org/t/question-about-coalesced-read-and-write-to-the-global-memory-using-cuda-jl-2d-grid/97689)

<div class="topic-metadata">

**Author:** [@fred\_wu](https://discourse.julialang.org/u/fred_wu)\
**Replies:** 1\
**Last updated:** [April 20, 2023, 8:02pm UTC](https://discourse.julialang.org/t/question-about-coalesced-read-and-write-to-the-global-memory-using-cuda-jl-2d-grid/97689 "2023-04-20T20:02:22Z")

</div>

Hi, I’m having some confusion with a simple test of matrix transpose using a 2D grid in CUDA.jl. I wrote two matrix transpose functions, the only difference is that in the first function, “ix” is along the direction of …

---

## [Efficient CuArray shift/rotation](https://discourse.julialang.org/t/efficient-cuarray-shift-rotation/86845)

<div class="topic-metadata">

**Author:** [@swibe](https://discourse.julialang.org/u/swibe)\
**Replies:** 2\
**Last updated:** [April 20, 2023, 7:55pm UTC](https://discourse.julialang.org/t/efficient-cuarray-shift-rotation/86845 "2023-04-20T19:55:53Z")

</div>

Hi all, I have a workload which regularly needs to perform operations on adjacent elements in an array and accumulate the result. I would sometimes like to be able to run this parallelised across the whole GPU, sometime…

---

## [GPU performance issues with an ML-from-scratch tutorial](https://discourse.julialang.org/t/gpu-performance-issues-with-an-ml-from-scratch-tutorial/97400)

<div class="topic-metadata">

**Author:** [@oschulz](https://discourse.julialang.org/u/oschulz)\
**Replies:** 7\
**Last updated:** [April 17, 2023, 7:39pm UTC](https://discourse.julialang.org/t/gpu-performance-issues-with-an-ml-from-scratch-tutorial/97400 "2023-04-17T19:39:59Z")

</div>

I put together (yet another) Julia ML-from-scratch tutorial for a machine learning course we held in Munich recently: GitHub - odsl-team/julia-ml-from-scratch: Machine learning from scratch in Julia It has very minimal …

---

## [Type instability with CuVector inside struct](https://discourse.julialang.org/t/type-instability-with-cuvector-inside-struct/97476)

<div class="topic-metadata">

**Author:** [@bjarthur](https://discourse.julialang.org/u/bjarthur)\
**Replies:** 2\
**Last updated:** [April 14, 2023, 5:45pm UTC](https://discourse.julialang.org/t/type-instability-with-cuvector-inside-struct/97476 "2023-04-14T17:45:38Z")

</div>

this is a very simple problem, so i fear i’m making a dumb mistake. apologies if so. but i can’t figure out why the function foo below is type unstable when input with a struct that has a CuVector in it, whereas if it’…

---

## [CUTENSOR not available](https://discourse.julialang.org/t/cutensor-not-available/97341)

<div class="topic-metadata">

**Author:** [@lello](https://discourse.julialang.org/u/lello)\
**Replies:** 7\
**Last updated:** [April 13, 2023, 9:36am UTC](https://discourse.julialang.org/t/cutensor-not-available/97341 "2023-04-13T09:36:02Z")

</div>

Hi All, I am trying to install CUDA with CUTENSOR support on a Debian 11+julia 1.8.5, but witout success. The main problem is that CUTENSOR is non supported at moment for the new CUDA 12.1 drivers. This is the error m…

---

## [Questions about using CUDA.jl for GPU concurrent programming: Computational results cannot be obtained when overlapping GPU and CPU operations](https://discourse.julialang.org/t/questions-about-using-cuda-jl-for-gpu-concurrent-programming-computational-results-cannot-be-obtained-when-overlapping-gpu-and-cpu-operations/97297)

<div class="topic-metadata">

**Author:** [@fred\_wu](https://discourse.julialang.org/u/fred_wu)\
**Replies:** 2\
**Last updated:** [April 12, 2023, 7:21am UTC](https://discourse.julialang.org/t/questions-about-using-cuda-jl-for-gpu-concurrent-programming-computational-results-cannot-be-obtained-when-overlapping-gpu-and-cpu-operations/97297 "2023-04-12T07:21:45Z")

</div>

Hi， I’m new to julia parallel programming.I want to make the idle CPU work together when the GPU is doing calculations, so I wrote the following code. However, what confuses me is that I can’t get the calculation result…

---

## [Indexing adjoints of CuArrays](https://discourse.julialang.org/t/indexing-adjoints-of-cuarrays/97215)

<div class="topic-metadata">

**Author:** [@MattSainsbury-Dale](https://discourse.julialang.org/u/MattSainsbury-Dale)\
**Replies:** 4\
**Last updated:** [April 10, 2023, 10:21pm UTC](https://discourse.julialang.org/t/indexing-adjoints-of-cuarrays/97215 "2023-04-10T22:21:14Z")

</div>

I was surprised by the following behaviour when indexing CuArrays: using CUDA CUDA.allowscalar(false) x = CUDA.fill(1.0f0, 3, 3) x\[1:2, :\] # works y = x' y\[1:2, :\] # error: scalar indexing is disallowed Is there a rea…

---

## [CUDA.jl crashes if a 4d FFT is asked](https://discourse.julialang.org/t/cuda-jl-crashes-if-a-4d-fft-is-asked/97165)

<div class="topic-metadata">

**Author:** [@shiroghost](https://discourse.julialang.org/u/shiroghost)\
**Replies:** 2\
**Last updated:** [April 7, 2023, 9:42pm UTC](https://discourse.julialang.org/t/cuda-jl-crashes-if-a-4d-fft-is-asked/97165 "2023-04-07T21:42:04Z")

</div>

Hi, It seems that CUDA.jl and FFTW.jl do not work for Arrays with dimensions larger than 3: julia\> b = randn(8, 8, 8); julia\> cb = CuArray(b); julia\> fft(b); julia\> fft(cb); julia\> sum(fft(b) .- Array(fft(cb))) 3.31956…

---

## [UndefVarError: libcuda\_original\_version not defined](https://discourse.julialang.org/t/undefvarerror-libcuda-original-version-not-defined/97049)

<div class="topic-metadata">

**Author:** [@stephans3](https://discourse.julialang.org/u/stephans3)\
**Replies:** 1\
**Last updated:** [April 4, 2023, 12:33pm UTC](https://discourse.julialang.org/t/undefvarerror-libcuda-original-version-not-defined/97049 "2023-04-04T12:33:37Z")

</div>

Hi all, on a student’s laptop (with Nvidia P620, Windows 10) we installed CUDA.jl and faced the following error message. julia\> CUDA.versioninfo() CUDA runtime 12.1, artifact installation CUDA driver 12.1 NVIDIA driver…

---

## [Complete and incomplete sparse cholesky factorization](https://discourse.julialang.org/t/complete-and-incomplete-sparse-cholesky-factorization/96918)

<div class="topic-metadata">

**Author:** [@shakedregev](https://discourse.julialang.org/u/shakedregev)\
**Replies:** 6\
**Last updated:** [April 4, 2023, 7:28am UTC](https://discourse.julialang.org/t/complete-and-incomplete-sparse-cholesky-factorization/96918 "2023-04-04T07:28:23Z")

</div>

Say I have a CuSparseCSR matrix “A” and I want to take complete and incomplete cholesky factorizations of it, but crucially leave that factorization on the GPU, how can I do that? Things I’ve tried: CuSparse.jl - depr…

---

## [Indexing in GPU kernel](https://discourse.julialang.org/t/indexing-in-gpu-kernel/96902)

<div class="topic-metadata">

**Author:** [@shakedregev](https://discourse.julialang.org/u/shakedregev)\
**Replies:** 2\
**Last updated:** [March 31, 2023, 7:07pm UTC](https://discourse.julialang.org/t/indexing-in-gpu-kernel/96902 "2023-03-31T19:07:11Z")

</div>

I’m building a custom GPU kernel and am having trouble indexing into a GPU vector. Q is a matrix stored in CSR format qrows - row pointers qcols - nonzero column indices qvals - nonzero values Here is a code snippet: …

---

## [Apple M1 GPU from Julia?](https://discourse.julialang.org/t/apple-m1-gpu-from-julia/69563)

<div class="topic-metadata">

**Author:** [@Per](https://discourse.julialang.org/u/Per)\
**Replies:** 20\
**Last updated:** [March 31, 2023, 8:49am UTC](https://discourse.julialang.org/t/apple-m1-gpu-from-julia/69563 "2023-03-31T08:49:13Z")

</div>

Does anyone know if it is possible to access the GPU of the Apple M1 chip from Julia? I really just need Float32 matrix-matrix multiplication (SGEMM). Anything else would be a bonus.

---

## [Sm90 (H100) support for cuda.jl](https://discourse.julialang.org/t/sm90-h100-support-for-cuda-jl/96809)

<div class="topic-metadata">

**Author:** [@smartalecH](https://discourse.julialang.org/u/smartalecH)\
**Replies:** 3\
**Last updated:** [March 30, 2023, 5:43pm UTC](https://discourse.julialang.org/t/sm90-h100-support-for-cuda-jl/96809 "2023-03-30T17:43:55Z")

</div>

Does CUDA.jl not yet support Hopper gpus (sm90)? I get the following error when trying to execute any kernels: ERROR: LoadError: CUDA error: device kernel image is invalid (code 300, ERROR\_INVALID\_SOURCE) Here’s my en…

---

## [Dealing with views and cuda array wrappers](https://discourse.julialang.org/t/dealing-with-views-and-cuda-array-wrappers/96725)

<div class="topic-metadata">

**Author:** [@smartalecH](https://discourse.julialang.org/u/smartalecH)\
**Replies:** 2\
**Last updated:** [March 29, 2023, 4:52pm UTC](https://discourse.julialang.org/t/dealing-with-views-and-cuda-array-wrappers/96725 "2023-03-29T16:52:18Z")

</div>

I’m working with an array wrapper that uses a CuArray under the hood. When I try to @view into it, I get a complicated SubArray that calls the Base.fill!() routine rather than the Cuda.fill!() routine (i.e. it’s scalar i…

---

## [GPU sum closure throwing an error](https://discourse.julialang.org/t/gpu-sum-closure-throwing-an-error/96658)

<div class="topic-metadata">

**Author:** [@weymouth](https://discourse.julialang.org/u/weymouth)\
**Replies:** 3\
**Last updated:** [March 28, 2023, 6:57am UTC](https://discourse.julialang.org/t/gpu-sum-closure-throwing-an-error/96658 "2023-03-28T06:57:36Z")

</div>

I’m trying to streamline some code using KernelAbstractions and it’s almost working perfectly. I’m getting hung up when I try to write a sum closure like this within the kernel using KernelAbstractions using CUDA u = CU…

---

## [(relative) newbie, looking for suggestions](https://discourse.julialang.org/t/relative-newbie-looking-for-suggestions/96677)

<div class="topic-metadata">

**Author:** [@skypuppy](https://discourse.julialang.org/u/skypuppy)\
**Replies:** 3\
**Last updated:** [March 28, 2023, 3:04am UTC](https://discourse.julialang.org/t/relative-newbie-looking-for-suggestions/96677 "2023-03-28T03:04:54Z")

</div>

“Pro” programmer since 1977, mostly with C and “Zim DBMS” (now defunct.) I want to play with neural networks now. Base hardware ultimate is Jetson Nano or Orin, for robot/drone tinkering and LIDAR image processing and …

---

## [MethodError: no method matching gemm!, It looks like |\>gpu cannot manage arrays resulting from view and reshape](https://discourse.julialang.org/t/methoderror-no-method-matching-gemm-it-looks-like-gpu-cannot-manage-arrays-resulting-from-view-and-reshape/96621)

<div class="topic-metadata">

**Author:** [@Andrea\_Deflorio](https://discourse.julialang.org/u/Andrea_Deflorio)\
**Replies:** 0\
**Last updated:** [March 26, 2023, 8:47am UTC](https://discourse.julialang.org/t/methoderror-no-method-matching-gemm-it-looks-like-gpu-cannot-manage-arrays-resulting-from-view-and-reshape/96621 "2023-03-26T08:47:55Z")

</div>

My julia version: 1.9 beta4 It looks like |\>gpu cannot manage arrays resulting from view and reshape. cu() can it! When I run the code down, I get this error: Exception has occurred: CompositeException TaskFailedExce…

---

## [LoadError: CUDA runtime not found](https://discourse.julialang.org/t/loaderror-cuda-runtime-not-found/96131)

<div class="topic-metadata">

**Author:** [@robvold](https://discourse.julialang.org/u/robvold)\
**Replies:** 11\
**Last updated:** [March 25, 2023, 1:57pm UTC](https://discourse.julialang.org/t/loaderror-cuda-runtime-not-found/96131 "2023-03-25T13:57:15Z")

</div>

I am trying to implement a couple of algorithms in CUDA using Julia and the CUDA.jl package. I started with Julia a couple of days ago, so the error may be easy to solve, but I haven’t figured out a solution yet. The Er…

---

## [CUDA Format of \_\_nvvm\_\_reflect function not recognized](https://discourse.julialang.org/t/cuda-format-of-nvvm-reflect-function-not-recognized/96370)

<div class="topic-metadata">

**Author:** [@Mark\_Lau](https://discourse.julialang.org/u/Mark_Lau)\
**Replies:** 12\
**Last updated:** [March 23, 2023, 4:19pm UTC](https://discourse.julialang.org/t/cuda-format-of-nvvm-reflect-function-not-recognized/96370 "2023-03-23T16:19:27Z")

</div>

I have some code that is similar to an example from the CUDA.jl Introduction. Specifically, this one: using CUDA N = 2^20 x\_d = CUDA.fill(1.0f0, N) # a vector stored on the GPU filled with 1.0 (Float32) y\_d = CUDA.fi…

---

## [Search in CUDA vector](https://discourse.julialang.org/t/search-in-cuda-vector/94833)

<div class="topic-metadata">

**Author:** [@MohHizzani](https://discourse.julialang.org/u/MohHizzani)\
**Replies:** 5\
**Last updated:** [March 23, 2023, 12:35pm UTC](https://discourse.julialang.org/t/search-in-cuda-vector/94833 "2023-03-23T12:35:46Z")

</div>

I have a vector of binary values, need to find all indices that are one, and randomly select one to update another vector at the same index selected.

---

## [LoadError: Could not find any suitable device for this configuration](https://discourse.julialang.org/t/loaderror-could-not-find-any-suitable-device-for-this-configuration/96253)

<div class="topic-metadata">

**Author:** [@stmorg](https://discourse.julialang.org/u/stmorg)\
**Replies:** 2\
**Last updated:** [March 20, 2023, 4:58pm UTC](https://discourse.julialang.org/t/loaderror-could-not-find-any-suitable-device-for-this-configuration/96253 "2023-03-20T16:58:02Z")

</div>

Hello, Currently setting up julia on my new machine and having some trouble with getting CUDA to work. I am getting errors during test. Below is the output with JULIA\_DEBUG=all Pkg.test("CUDA") ┌ Debug: Loading cache f…

---

## [MPI RMA with CUDA](https://discourse.julialang.org/t/mpi-rma-with-cuda/96219)

<div class="topic-metadata">

**Author:** [@smartalecH](https://discourse.julialang.org/u/smartalecH)\
**Replies:** 2\
**Last updated:** [March 17, 2023, 4:47am UTC](https://discourse.julialang.org/t/mpi-rma-with-cuda/96219 "2023-03-17T04:47:21Z")

</div>

I’m interested in using MPI RMA (i.e. one-sided operations) with native CuArrays. From the MPI.jl documentation, we read: \[CuArrays\] may also work with one-sided operations, but these are not often supported Could s…

---

## [AMDGPU.jl status](https://discourse.julialang.org/t/amdgpu-jl-status/71191)

<div class="topic-metadata">

**Author:** [@torrance](https://discourse.julialang.org/u/torrance)\
**Replies:** 11\
**Last updated:** [March 16, 2023, 1:46pm UTC](https://discourse.julialang.org/t/amdgpu-jl-status/71191 "2023-03-16T13:46:10Z")

</div>

I’m beginning a fairly large project that is destined to run on a newly commissioned HPC cluster that will be using AMD’s Milan CPUs and MI250 GPUs. Once we have proven the algorithm works on the CPU it will be ported to…

---

## [Is accumulate() with '+' producing the wrong results?](https://discourse.julialang.org/t/is-accumulate-with-producing-the-wrong-results/95952)

<div class="topic-metadata">

**Author:** [@P570](https://discourse.julialang.org/u/P570)\
**Replies:** 1\
**Last updated:** [March 13, 2023, 11:06am UTC](https://discourse.julialang.org/t/is-accumulate-with-producing-the-wrong-results/95952 "2023-03-13T11:06:25Z")

</div>

Hi, I lately started experimenting a bit with CUDA and was doing some exploratory work through the REPL. I was doing some addition tests with accumulate() and reduce() when I stumbled on what seems to be an issue with …

[Previous page](https://discourse.julialang.org/c/domain/gpu/11.md?page=11)

[Next page](https://discourse.julialang.org/c/domain/gpu/11.md?page=13)