We’ve been working on a new 100% Julia backend for GPU accelerated raytracing for Makie to replace RPRMakie!
Its physically based ray tracing (pbrt) and is pretty much a Julia port of the amazing pbrt-v4 ray tracer.
I also added an experimental rasterizer, which also runs on the GPU via KernelAbstractions - which could pave a way to have a 100% Julia GPU graphics API, which has the potential to rival OpenGL in the future (still pretty hypothetical, we will need to see how it matches performance and how well it works across platforms).
I just want to say, that I’m really happy with how the GPU stack has matured and how it has become possible to write this while reaching performance which is in the same regime as established ray tracers!
Read all the details in https://makie.org/website/blogposts/raytracing/.
I have tried Metal many times while Simon was developing this and it never worked quite right. Not sure about the current state, last I tried was two weeks ago or so. Either the code would not compile at all, because Metal is a bit stricter than the other backends with requiring Float32 types. And when it compiled and returned an image, there was a weird issue that I assume is because of some race condition, maybe related to atomics? The visual effect was that objects looked more or less transparent because in each sample, they’d sometimes be hit by rays and sometimes not. This would even happen after I removed all random number generation from the kernels, so rays should have been completely deterministic across samples. I was not able to debug this yet, I dug in to a point where I could see nondeterministic behavior in the queue sizes but not where it ultimately stemmed from. Simon recommended to try and build up a simpler example of the problem than starting with the full tracer but I didn’t have time for that.
To test, you have to pass Metal.MtlArray as the backend type to the integrator.
Yeah, I hope we figure out Metal support soon!
I do bet a bit on claude code, which is pretty good at debugging these kind of problems when it is allowed to execute Julia code.
Usually its enough to dev all the involved packages and set it on one of the example scenes with a low resolution and sample count and tell it to fix all errors it encounters. It may also be required to guide it to towards creating MWEs until it has isolated the problems for harder problems (e.g. like the atomic operation bug).
Btw, the API has changed, so now one needs to pass device=Metal.MetalBackend().
In theory, yes!
In practice quite a lot would be required to get julia to compile to SPIR-V and have the RT core extensions available.
I do hope that the GPU infrastructure matures in that direction and I do think we have most of the puzzle pieces together for this (we do already compile to SPIR-V e.g. in OpenCL.jl).
Note that OpenCL is a different SPIR-V dialect than GLSL/Vulkan with stricter semantics regarding control flow etc., which complicates things a bit (there are already issues with the OpenCL dialect not supporting the way Julia throws exceptions for example). There has recently been some work on supporting GLSL SPIR-V in the LLVM backend, which might help, but I believe SPIRV.jl even decided to skip LLVM entirely
Also, before the API changed, I was able get things alright. I remember I had to unroll the for loops in the kernels, otherwise all was messy and most of the times black. We will need to start again and see. I should have the MWE around, somewhere.
