RFC: Mitsuba2.jl - differentiable physics-based renderer

Specific Domains Modelling & Simulations
1

Hey all, I wanted to share a package I started, and I am hoping to get feedback from anyone who is interested in physics-based rendering. My package, Mitsuba2.jl, brings the Mitsuba 2 renderer into Julia via its Python interface and PyCall.jl.

using Mitsuba2

# choose variant (GPU, RGB vs spectral, autodiff, etc.)
import Mitsuba2.scalar_rgb as mitsuba

scene = mitsuba.load_string(read("scene.xml", String))
sensor = first(scene.sensors())
scene.integrator().render(scene, sensor)
bmp = sensor.film().bitmap(raw=true)

I have quite a few ideas for how to improve this pre-alpha package. So far, this package shows promise as a high-level wrapper. Some TODOs and ideas include the following:

  • test on more Linux distributions (status: not started)
  • assess compatibility with Windows / Mac OS (status: not started)
  • bring in binaries more easily via BinaryBuilder.jl / Yggdrasil
  • improve the way symbols are put into modules, potentially leveraging the way the Python bindings are generated by upstream (status: not started)
  • dynamically generate scene files using structs via AcuteML.jl (status: mocked-up locally)
  • integrate with Flux.jl (status: not started)
  • consider a refactor into a CxxWrap.jl or Cxx.jl based binding (status: not sure if desirable)
  • achieve easy maintainability and offer this work to up-stream (status: not sure if this desired, idiomatic, or even possible)

A few specific things Iā€™m looking for feedback on:

  • ideas are welcomed for further improvement
  • how idiomatic is it to generate modules in the current way? concerned about precompilation (seems to work on 1.7.3 though)
  • does this even work for anyone else yet? (only tested on Arch Linux)

I am waiting to register this until the new version of Mitsuba 2 becomes available (around SIGGRAPH/August) and the package becomes more mature. Thank you for your attention!

8 Likes