[blog post] Implement You Own Source to Source AD in One day

Community
1

This is a blog post I wanted to write since last one :wink:

http://blog.rogerluo.me/2019/07/27/yassad/

And I did it in JuliaCon2019 Hackthon with the help of @MikeInnes If you are interested in the internal of Zygote, this might be a good reference for you! and after you read this I’m sure the Zygote won’t look like black magic to you anymore! (so you can help fix things as well)

Bests
Roger

28 Likes
2

Thanks a lot for the invaluable post!

You mention an implementation of AD in Casette, how does this compare to YASSAD (and Zygote)? Is this also source to source? Why not just use Casette for the contextual dispatch?

Thanks,
Jules

1 Like
3

I think contextual dispatch is required for tracing the operations, but to implement an AD (with branches potentially) you also need to implement a generic pullback by manipulating the IR (which inverse the call stack and create the adjoint of control flow) which is also possible with Cassette’s Reflection, contextual tagging etc.

In general to implement a source 2 source AD in Julia, one can do that with bare hands, but IRTools as it is named is just a tool to help you manipulate the IR. It seems more straightforward to help people understand how Zygote works. (I’m not going to maintain and implement a brand new AD)

2 Likes