[WIP] Diff2DRaster.jl - Differentiable 2d graphics rasterizer

I am working on a differentiable 2D rasterizer Diff2DRaster.jl which uses Zygote to calculate the gradient of geometric parameters, allowing some basic inverse rendering of 2D graphics.

The code uses signed distance functions (SDFs) to define geometric primitives, and by using a 1d anti-aliasing kernel orthogonal to the implicit curve defined by the SDF, it achieves continuous derivatives from parameters to pixel values.

To work with Zygote the rendering is done without state mutation. The forward pass of the rendering is quite fast with few dynamical allocations, but the gradient is more expensive to run, typically several orders of magnitude more allocations. I am new to Julia and Zygote and still trying to figure out how the performance can be improved.

This is work in progress and not yet a publishable package, but as a working demonstration differentiable graphics/inverse rendering it may already be useful for someone. My own use-case have been recreational vector art generation, but I am very open for suggestions for which parts could be useful as a standalone library.

Jan Tore

1 Like