# RFC: Mitsuba2.jl - differentiable physics-based renderer

**URL:** <https://discourse.julialang.org/t/rfc-mitsuba2-jl-differentiable-physics-based-renderer/82879>\
**Category:** Modelling & Simulations\
**Created:** [June 16, 2022, 2:21pm UTC](https://discourse.julialang.org/t/rfc-mitsuba2-jl-differentiable-physics-based-renderer/82879 "2022-06-16T14:21:24Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![alhirzel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alhirzel/32/35708_2.png) [@alhirzel](https://discourse.julialang.org/u/alhirzel)\
**Post date:** [June 16, 2022, 2:21pm UTC](https://discourse.julialang.org/t/rfc-mitsuba2-jl-differentiable-physics-based-renderer/82879/1 "2022-06-16T14:21:24Z")

</div>

Hey all, I wanted to share a package I started, and I am hoping to get feedback from anyone who is interested in [physics-based rendering](https://www.pbr-book.org/3ed-2018/contents). My package, [Mitsuba2.jl](https://github.com/alhirzel/Mitsuba2.jl), brings [the Mitsuba 2 renderer](http://rgl.epfl.ch/publications/NimierDavidVicini2019Mitsuba2) into Julia via [its Python interface](https://mitsuba2.readthedocs.io/en/latest/src/python_interface/intro.html) and [PyCall.jl](https://github.com/JuliaPy/PyCall.jl).

```julia
using Mitsuba2

# choose variant (GPU, RGB vs spectral, autodiff, etc.)
import Mitsuba2.scalar_rgb as mitsuba

scene = mitsuba.load_string(read("scene.xml", String))
sensor = first(scene.sensors())
scene.integrator().render(scene, sensor)
bmp = sensor.film().bitmap(raw=true)

```

![](https://global.discourse-cdn.com/julialang/original/3X/4/a/4a18551cfb9baa491f945c752ebbcfa3a926113e.png)

I have quite a few ideas for how to improve this pre-alpha package. So far, this package shows promise as a high-level wrapper. Some TODOs and ideas include the following:

- test on more Linux distributions _(status: not started)_
- assess compatibility with Windows / Mac OS _(status: not started)_
- bring in binaries more easily via [BinaryBuilder.jl](https://github.com/JuliaPackaging/BinaryBuilder.jl) / [Yggdrasil](https://github.com/JuliaPackaging/Yggdrasil)
- improve the way symbols are put into modules, potentially leveraging [the way the Python bindings are generated by upstream](https://github.com/mitsuba-renderer/mitsuba2/tree/master/src/librender/python) _(status: not started)_
- dynamically generate scene files using `struct`s via [AcuteML.jl](https://github.com/aminya/AcuteML.jl) _(status: mocked-up locally)_
- integrate with [Flux.jl](https://github.com/FluxML/Flux.jl) _(status: not started)_
- consider a refactor into a [CxxWrap.jl](https://github.com/JuliaInterop/CxxWrap.jl) or [Cxx.jl](https://github.com/JuliaInterop/Cxx.jl) based binding _(status: not sure if desirable)_
- achieve easy maintainability and offer this work to up-stream _(status: not sure if this desired, idiomatic, or even possible)_

A few specific things I’m looking for feedback on:

- ideas are welcomed for further improvement
- how idiomatic is it to generate modules in the current way? concerned about precompilation (seems to work on 1.7.3 though)
- does this even work for anyone else yet? (only tested on Arch Linux)

I am waiting to register this until the new version of Mitsuba 2 becomes available (around SIGGRAPH/August) and the package becomes more mature. Thank you for your attention!