[ANN] Ghost.jl - the code tracer

Package Announcements
1

Ghost.jl lets you recursively trace a function execution and record all primitive operations onto a linearized tape.

using Ghost

inc(x) = x + 1
mul(x, y) = x * y
inc_double(x) = mul(inc(x), inc(x))

val, tape = trace(inc_double, 2.0)

result:

(9.0, Tape{Dict{Any, Any}}
  inp %1::typeof(Main.__atexample__257.inc_double)
  inp %2::Float64
  %3 = +(%2, 1)::Float64
  %4 = +(%2, 1)::Float64
  %5 = *(%3, %4)::Float64
)

Previously Ghost has been part of the autodiff package Yota and was used to break down complex functions into the list of primitives amenable to differentiation. Other use cases include function serialization, analysis or modification, as well as providing a general graph representation.

As an experimental feature, Ghost also supports loop operation, making it suitable for many dynamic graph problems.

Ghost has some intersecting functionality with IR-level packages such as Cassette, IRTools, CodeInfoTools, etc., but works on a higher level (i.e. no segfaults due to incorrectly inserted operation) and makes emphasis on linearized graph representation.

14 Likes
2

This looks cool. I’ve also seen GitHub - MasonProtter/SymbolicTracing.jl: (experimental) tracing and abstract interpretation with SymbolicUtils.jl. Do you or @Mason know how these relate to each other?

3

SymbolicTracing looks interesting and the two packages certainly have similar approaches to tracing. A few differences I see though (based on the README):

  • SymbolicTracing works only with abstract type while Ghost can as well trace through concrete types and struct fields
  • SymbolicTracing doesn’t seem to handle control flow; Ghost can follow branches, including conditions and loops
  • Loops in Ghost can also be traced as a separate operation
  • I’m not sure it’s possible to include operations out of the dependency graph in SymbolicTracing (e.g. println() statement); in Ghost you can have both - a pure computational graph and a graph with side effects
  • In general, Symbolic* packages seem to be more oriented to pure symbolic graphs, while Ghost’s Tape is an abstraction for any kind of computation
3 Likes
4

Is there a way to get a pullback function with Yota (for vector-jacobian-products), instead of “just” a gradient?

Posted to the latest Yotal.jl announcement instead.

5

Having written an IRTools-based tracker myself, I must say that the internals look really nice and clean!

3 Likes