# RayTracer.jl a differentiable ray tracer

**URL:** <https://discourse.julialang.org/t/raytracer-jl-a-differentiable-ray-tracer/26527>\
**Category:** Machine Learning\
**Created:** [July 19, 2019, 3:32am UTC](https://discourse.julialang.org/t/raytracer-jl-a-differentiable-ray-tracer/26527 "2019-07-19T03:32:21Z")\
**Posts on this page:** 10\
**Page:** 2

<div class="post-metadata">

**Author:** ![yakir12](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yakir12/32/297_2.png) [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Post date:** [July 21, 2019, 8:36am UTC](https://discourse.julialang.org/t/raytracer-jl-a-differentiable-ray-tracer/26527/21 "2019-07-21T08:36:46Z")

</div>

What with the [publication](https://github.com/JuliaCon/proceedings-papers/blob/jcon.00037/jcon.00037/10.21105.jcon.00037.pdf) in JuliaCon, I can’t help wondering if this thing can be used to calibrate actual cameras. In my field we record animal behavior in the field or in the lab using a video camera. To convert the pixel coordinates of the trajectory or location of the animal to real-world coordinates (i.e. in cm), we first calibrate the camera with a checkerboard (see [here](http://www.vision.caltech.edu/bouguetj/calib_doc/) for instance). This is a bit of an involved process that I’d love to improve. I’m wondering if this can some how improve on that…

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [July 21, 2019, 10:41am UTC](https://discourse.julialang.org/t/raytracer-jl-a-differentiable-ray-tracer/26527/22 "2019-07-21T10:41:24Z")

</div>

This post was temporarily hidden by the community for possibly being off-topic, unfocused, inappropriate, or spammy.

---

<div class="post-metadata">

**Author:** ![avikpal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/avikpal/32/6550_2.png) [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Post date:** [July 26, 2019, 1:21am UTC](https://discourse.julialang.org/t/raytracer-jl-a-differentiable-ray-tracer/26527/23 "2019-07-26T01:21:30Z")

</div>

I don’t think there is much literature that explores this particular idea. However, I gave a talk on this topic at JuliaCon. The video is [here](https://www.youtube.com/watch?v=cdwLJCb45Kk&t=1s).

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [July 26, 2019, 4:19pm UTC](https://discourse.julialang.org/t/raytracer-jl-a-differentiable-ray-tracer/26527/24 "2019-07-26T16:19:31Z")

</div>

Thanks I saw the talk and I am sold on the idea, great work 🙂

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [August 8, 2019, 10:16am UTC](https://discourse.julialang.org/t/raytracer-jl-a-differentiable-ray-tracer/26527/25 "2019-08-08T10:16:44Z")

</div>

Updating this thread. A paper on an Non Line Of Sight camerasystem was presented at SIGGRAPH

[https://sciencenode.org/feature/Seeing%20around%20corners.php](https://sciencenode.org/feature/Seeing%20around%20corners.php)

> **[Computational Imaging Wave-Based Non-Line-of-Sight Imaging using Fast f-k...](http://www.computationalimaging.org/publications/nlos-fk/)**

Github with Matlab code here: [GitHub - computational-imaging/nlos-fk: Processing code for "Wave-Based Non-Line-of-Sight Imaging using Fast f-k Migration"](https://github.com/computational-imaging/nlos-fk)

---

<div class="post-metadata">

**Author:** ![yakir12](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yakir12/32/297_2.png) [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Post date:** [August 22, 2019, 9:31am UTC](https://discourse.julialang.org/t/raytracer-jl-a-differentiable-ray-tracer/26527/26 "2019-08-22T09:31:09Z")

</div>

@avikpal started: [Calibrating Camera Parameters - Nextjournal](https://nextjournal.com/avikpal/calibrating-camera-parameters)

This is awesome!!! Hope this matures to a camera calibrating toolbox!!!

---

<div class="post-metadata">

**Author:** ![krrutkow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/krrutkow/32/9138_2.png) [@krrutkow](https://discourse.julialang.org/u/krrutkow)\
**Post date:** [August 26, 2019, 3:31pm UTC](https://discourse.julialang.org/t/raytracer-jl-a-differentiable-ray-tracer/26527/27 "2019-08-26T15:31:52Z")

</div>

@johnh I actually recently posted a Julia project regarding the nlos-fk paper/code you linked to.

[https://github.com/analytech-solutions/fkMigration.jl](https://github.com/analytech-solutions/fkMigration.jl)

---

<div class="post-metadata">

**Author:** ![winterismute](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/winterismute/32/13976_2.png) [@winterismute](https://discourse.julialang.org/u/winterismute)\
**Post date:** [April 9, 2020, 6:22pm UTC](https://discourse.julialang.org/t/raytracer-jl-a-differentiable-ray-tracer/26527/28 "2020-04-09T18:22:56Z")

</div>

Hi there,  
This is a really interesting project, I am new to Julia but I wanted to give it ago hence I have installed everything yesterday, however when I try to run one of the examples I get the following errors:

```julia
~/projects/raytracer.jil/examples$ julia --project=. --color=yes teapot_rendering.jl 
┌ Warning: Package MacroTools does not have Markdown in its dependencies:
│ - If you have MacroTools checked out for development and have
│ added Markdown as a dependency but haven't updated your primary
│ environment's manifest file, try `Pkg.resolve()`.
│ - Otherwise you may need to report an issue with MacroTools
└ Loading Markdown into MacroTools from project dependency, future warnings for MacroTools are suppressed.
┌ Warning: The call to compilecache failed to create a usable precompiled cache file for ZygoteRules [700de1a5-db45-46bc-99cf-38207098b444]
│ exception = Required dependency MacroTools [1914dd2f-81c6-5fcd-8719-6d5c9610ff09] failed to load from a cache file.
└ @ Base loading.jl:1041
┌ Warning: Package MacroTools does not have Markdown in its dependencies:
│ - If you have MacroTools checked out for development and have
│ added Markdown as a dependency but haven't updated your primary
│ environment's manifest file, try `Pkg.resolve()`.
│ - Otherwise you may need to report an issue with MacroTools
└ Loading Markdown into MacroTools from project dependency, future warnings for MacroTools are suppressed.
┌ Warning: Package Requires does not have UUIDs in its dependencies:
│ - If you have Requires checked out for development and have
│ added UUIDs as a dependency but haven't updated your primary
│ environment's manifest file, try `Pkg.resolve()`.
│ - Otherwise you may need to report an issue with Requires
└ Loading UUIDs into Requires from project dependency, future warnings for Requires are suppressed.
ERROR: LoadError: ArgumentError: Package SpecialFunctions does not have OpenSpecFun_jll in its dependencies:
- If you have SpecialFunctions checked out for development and have
  added OpenSpecFun_jll as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with SpecialFunctions
Stacktrace:
 [1] require(::Module, ::Symbol) at ./loading.jl:905
 [2] include(::Module, ::String) at ./Base.jl:377
 [3] top-level scope at none:2
 [4] eval at ./boot.jl:331 [inlined]
 [5] eval(::Expr) at ./client.jl:449
 [6] top-level scope at ./none:3
in expression starting at /Users/rbonfiglioli/.julia/packages/SpecialFunctions/5QYEE/src/SpecialFunctions.jl:3
ERROR: LoadError: LoadError: Failed to precompile SpecialFunctions [276daf66-3868-5448-9aa4-cd146d93841b] to /Users/rbonfiglioli/.julia/compiled/v1.4/SpecialFunctions/78gOt_IFofr.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
 [3] _require(::Base.PkgId) at ./loading.jl:1029
 [4] require(::Base.PkgId) at ./loading.jl:927
 [5] require(::Module, ::Symbol) at ./loading.jl:922
 [6] include(::Module, ::String) at ./Base.jl:377
 [7] include(::String) at /Users/rbonfiglioli/.julia/packages/Zygote/iOhwI/src/Zygote.jl:1
 [8] top-level scope at /Users/rbonfiglioli/.julia/packages/Zygote/iOhwI/src/Zygote.jl:24
 [9] include(::Module, ::String) at ./Base.jl:377
 [10] top-level scope at none:2
 [11] eval at ./boot.jl:331 [inlined]
 [12] eval(::Expr) at ./client.jl:449
 [13] top-level scope at ./none:3
in expression starting at /Users/rbonfiglioli/.julia/packages/Zygote/iOhwI/src/lib/number.jl:1
in expression starting at /Users/rbonfiglioli/.julia/packages/Zygote/iOhwI/src/Zygote.jl:24
ERROR: LoadError: Failed to precompile Zygote [e88e6eb3-aa80-5325-afca-941959d7151f] to /Users/rbonfiglioli/.julia/compiled/v1.4/Zygote/4kbLI_IFofr.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
 [3] _require(::Base.PkgId) at ./loading.jl:1029
 [4] require(::Base.PkgId) at ./loading.jl:927
 [5] require(::Module, ::Symbol) at ./loading.jl:922
 [6] include(::Module, ::String) at ./Base.jl:377
 [7] top-level scope at none:2
 [8] eval at ./boot.jl:331 [inlined]
 [9] eval(::Expr) at ./client.jl:449
 [10] top-level scope at ./none:3
in expression starting at /Users/rbonfiglioli/.julia/packages/RayTracer/VcDKP/src/RayTracer.jl:3
ERROR: LoadError: Failed to precompile RayTracer [60dacb86-48ff-11e9-0f01-03ab8794bbc9] to /Users/rbonfiglioli/.julia/compiled/v1.4/RayTracer/sUryZ_IFofr.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
 [3] _require(::Base.PkgId) at ./loading.jl:1029
 [4] require(::Base.PkgId) at ./loading.jl:927
 [5] require(::Module, ::Symbol) at ./loading.jl:922
 [6] include(::Module, ::String) at ./Base.jl:377
 [7] exec_options(::Base.JLOptions) at ./client.jl:288
 [8] _start() at ./client.jl:484
in expression starting at /Users/rbonfiglioli/projects/raytracer.jil/examples/teapot_rendering.jl:11

```

Mayve @avikpal himself has a hint? I am on OSX Mojave and installed Julia 1.4

---

<div class="post-metadata">

**Author:** ![avikpal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/avikpal/32/6550_2.png) [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Post date:** [April 9, 2020, 7:48pm UTC](https://discourse.julialang.org/t/raytracer-jl-a-differentiable-ray-tracer/26527/29 "2020-04-09T19:48:48Z")

</div>

Hi @winterismute, this issue has been fixed on master, and will be soon reflected on the tagged version.

---

<div class="post-metadata">

**Author:** ![Phil\_Tomson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/phil_tomson/32/3575_2.png) [@Phil\_Tomson](https://discourse.julialang.org/u/Phil_Tomson)\
**Post date:** [April 15, 2020, 6:25pm UTC](https://discourse.julialang.org/t/raytracer-jl-a-differentiable-ray-tracer/26527/30 "2020-04-15T18:25:03Z")

</div>

I’m seeing this error when I try to run:

> $ julia --project=. --color=yes teapot\_rendering.jl  
> ┌ Warning: Package Colors does not have Printf in its dependencies:  
> │ - If you have Colors checked out for development and have  
> │ added Printf as a dependency but haven’t updated your primary  
> │ environment’s manifest file, try `Pkg.resolve()`.  
> │ - Otherwise you may need to report an issue with Colors  
> └ Loading Printf into Colors from project dependency, future warnings for Colors are suppressed.  
> ┌ Warning: The call to compilecache failed to create a usable precompiled cache file for Flux [587475ba-b771-5e3f-ad9e-33799f191a9c]  
> │ exception = Required dependency Colors [5ae59095-9a9b-59fe-a467-6f913c188581] failed to load from a cache file.  
> └ @ Base loading.jl:1041  
> ┌ Warning: Package Colors does not have Printf in its dependencies:  
> │ - If you have Colors checked out for development and have  
> │ added Printf as a dependency but haven’t updated your primary  
> │ environment’s manifest file, try `Pkg.resolve()`.  
> │ - Otherwise you may need to report an issue with Colors  
> └ Loading Printf into Colors from project dependency, future warnings for Colors are suppressed.  
> WARNING: ImageQualityIndexes.psnr is deprecated, use assess\_psnr instead.  
> likely near /home/phil/.julia/packages/Images/0SDGT/src/Images.jl:60  
> ┌ Warning: Package RayTracer does not have CUDAnative in its dependencies:  
> │ - If you have RayTracer checked out for development and have  
> │ added CUDAnative as a dependency but haven’t updated your primary  
> │ environment’s manifest file, try `Pkg.resolve()`.  
> │ - Otherwise you may need to report an issue with RayTracer  
> └ Loading CUDAnative into RayTracer from project dependency, future warnings for RayTracer are suppressed.  
> ERROR: LoadError: LoadError: LoadError: UndefVarError: **@nograd not defined**  
> Stacktrace:  
> [1] top-level scope  
> [2] include(::Module, ::String) at ./Base.jl:377  
> [3] include(::String) at /home/phil/.julia/packages/RayTracer/rEd4h/src/RayTracer.jl:1  
> [4] top-level scope at /home/phil/.julia/packages/RayTracer/rEd4h/src/RayTracer.jl:24  
> [5] include(::Module, ::String) at ./Base.jl:377  
> [6] top-level scope at none:2  
> [7] eval at ./boot.jl:331 [inlined]  
> [8] eval(::Expr) at ./client.jl:449  
> [9] top-level scope at ./none:3

[Previous page](https://discourse.julialang.org/t/raytracer-jl-a-differentiable-ray-tracer/26527.md?page=1)