How do I do real-time rendering with CUDA.jl?

General Usage Performance
, , ,
1

I’ve spent a couple of hours looking around online and haven’t found a good solution, so I think I’ve put in the effort necessary to justify asking here. How would one easily go about taking something like a CuArray from CUDA.jl and displaying it in a window? I know that I can convert it into a regular array and use Image.jl and Plots.jl to display it, but that gives me a window with lots of extra stuff around it and is very slow because I have to copy all of the data over to the CPU every frame. I would like to simply have a window the same size as my array that can be updated using that array in real-time (144hz for me). Thanks in advance for any help or advice provided.

EDIT: From the research I’ve done and the resources people have provided, I’ve concluded that Julia+CUDA is not the correct set of tools for real-time rendering.

1 Like
2

this is not something you use CUDA for, you’re asking Computational Graphics question, which I think the only GPU accelerated plotting (“rendering”) library is Makie.jl, specifically, GLMakie.jl, but idk if it’s optimized for speed to the point you can do 144hz, technically it should be possible since it’s OpenGL powered

1 Like
3

What I want to make can’t be done efficiently using traditional graphics pipelines. If I used a normal graphics API, I would be writing almost exclusively compute shaders. I know that CUDA can’t display things on a window on its own, which is why I’m asking for a good way to do that. I don’t want to use GLMakie because it’s a plotting tool and won’t give the output that I want.

5

Mmm! I’m not sure if there is anything out of the box, unless you can find someone that’s done something very similar.

For high performance CPU->GPU in C, I found that this technique of mapping onto a GL texture with your array, and then projecting onto a window worked very well:
https://stackoverflow.com/a/504425

I imagine something similar could be built with the Julia GL ecosystem:

6

do you consider all the game engines “traditional graphics pipelines”? in any case, we don’t have a game engine in Julia :slight_smile:

7

GLMakie is really quite flexible and shouldn’t be discounted out of hand (especially if you’re just trying to display a 2D array as an image or heatmap in a loop). Take a look at this post for GLMakie/CUDA integration without any CPU-GPU data transfer: CuArray + GLMakie - #8 by maleadt

1 Like