# ReverseDiff.jl

**URL:** <https://discourse.julialang.org/t/reversediff-jl/764>\
**Category:** Community\
**Tags:** package, announcement\
**Created:** [December 6, 2016, 3:53pm UTC](https://discourse.julialang.org/t/reversediff-jl/764 "2016-12-06T15:53:26Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![jrevels](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jrevels/32/10393_2.png) [@jrevels](https://discourse.julialang.org/u/jrevels)\
**Post date:** [December 6, 2016, 3:53pm UTC](https://discourse.julialang.org/t/reversediff-jl/764/1 "2016-12-06T15:53:26Z")

</div>

Hi all,

I’m pleased to announce that the first version of [ReverseDiff](https://github.com/JuliaDiff/ReverseDiff.jl) is now registered. ReverseDiff is brought to you by the same folks who develop and maintain the ForwardDiff package.

ReverseDiff implements methods to take **gradients** , **Jacobians** , **Hessians** , and higher-order derivatives of native Julia functions (or any callable object, really) using **reverse mode automatic differentiation (AD)**. While performance can vary depending on the functions you evaluate, the algorithms implemented by ReverseDiff **generally outperform non-AD algorithms in both speed and accuracy.**

Why use ReverseDiff as your go-to reverse-mode AD package? Here are some reasons:

- supports a large subset of the Julia language, including loops, recursion, and control flow
- user-friendly API for reusing and compiling tapes
- user-friendly performance annotations such as `@forward` and `@skip` (with more to come!)
- compatible with ForwardDiff, enabling mixed-mode AD
- built-in definitions leverage the benefits of ForwardDiff’s `Dual` numbers (e.g. SIMD, zero-overhead arithmetic)
- a familiar differentiation API for ForwardDiff users
- non-allocating linear algebra optimizations
- nested differentiation
- suitable as an execution backend for graphical machine learning libraries
- ReverseDiff doesn’t need to record scalar indexing operations (a huge cost for many similar libraries)
- higher-order `map` and `broadcast` optimizations
- it’s well tested

In the future, we aim to add GPU support, sparsity exploitation, and more optimized linear algebra derivatives to the package.

Compared to ForwardDiff, ReverseDiff’s methods are **algorithmically more efficient for differentiating functions where the input dimension is larger than the output dimension**. In general, the optimal choice between ForwardDiff and ReverseDiff for a given problem requires some nuance, so I’ve [written up some tips to guide your choice](https://github.com/JuliaDiff/ReverseDiff.jl#should-i-use-reversediff-or-forwarddiff).

Internally, ReverseDiff contains facilities for recording execution traces of native Julia code to reusable, compilable instruction tapes, as well as mechanisms for propagating values “forwards” and “backwards” through these tapes. Since these tapes can be analyzed as computation graphs, my hope is that this infrastructure can eventually be rendered useful for non-AD purposes, such as performance optimization, scheduled parallel execution, and constraint programming. Feel free to reach out if you’re interested in exploring this area!

---

<div class="post-metadata">

**Author:** ![ahwillia](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ahwillia/32/105_2.png) [@ahwillia](https://discourse.julialang.org/u/ahwillia)\
**Post date:** [December 6, 2016, 4:56pm UTC](https://discourse.julialang.org/t/reversediff-jl/764/2 "2016-12-06T16:56:03Z")

</div>

Congrats @jrevels - This is exciting!

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [December 6, 2016, 5:45pm UTC](https://discourse.julialang.org/t/reversediff-jl/764/3 "2016-12-06T17:45:33Z")

</div>

This is awesome! ForwardDiff.jl is already one of my favorite parts of the Julia ecosystem, and I’m thrilled to be able to try out reverse-mode just as easily.

---

<div class="post-metadata">

**Author:** ![MikeInnes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikeinnes/32/3656_2.png) [@MikeInnes](https://discourse.julialang.org/u/MikeInnes)\
**Post date:** [December 6, 2016, 6:46pm UTC](https://discourse.julialang.org/t/reversediff-jl/764/4 "2016-12-06T18:46:45Z")

</div>

Congrats on the announcement @jrevels! This is very cool, the benchmark results are extremely impressive, and I like the possibilities of the instruction tape work. Can’t wait to see what people build on top of this.

---

<div class="post-metadata">

**Author:** ![cortner](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cortner/32/204_2.png) [@cortner](https://discourse.julialang.org/u/cortner)\
**Post date:** [December 6, 2016, 9:23pm UTC](https://discourse.julialang.org/t/reversediff-jl/764/5 "2016-12-06T21:23:17Z")

</div>

when trying to add the package I get

```julia
ERROR: unsatisfiable package requirements detected: no feasible version could be found for package: DiffBase

```

any suggestion what might have gone wrong?

---

<div class="post-metadata">

**Author:** ![jrevels](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jrevels/32/10393_2.png) [@jrevels](https://discourse.julialang.org/u/jrevels)\
**Post date:** [December 6, 2016, 9:46pm UTC](https://discourse.julialang.org/t/reversediff-jl/764/6 "2016-12-06T21:46:46Z")

</div>

Here’s [ReverseDiff’s REQUIRE file](https://github.com/JuliaDiff/ReverseDiff.jl/blob/master/REQUIRE); it seems that you have a package somewhere that conflicts with these version requirements?

For example, ReverseDiff requires ForwardDiff 0.3.2 or higher, but a lot of popular packages (like JuMP and Optim) haven’t made the switch over yet (they’ll be switched over soon, though).

I’m not really sure why you’d see that error with DiffBase, though - the only packages I know of that use it currently are ForwardDiff and ReverseDiff, and neither of them have upper bounds on it.

---

<div class="post-metadata">

**Author:** ![jmxpearson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmxpearson/32/3819_2.png) [@jmxpearson](https://discourse.julialang.org/u/jmxpearson)\
**Post date:** [December 6, 2016, 10:20pm UTC](https://discourse.julialang.org/t/reversediff-jl/764/7 "2016-12-06T22:20:34Z")

</div>

Hooray! For those who haven’t looked at the code, it’s really impressive how much @jrevels has managed to generalize and optimize the algorithm to support a broad base of Julia functionality.

This functionality should be really helpful to those implementing Julia machine learning algorithms going forward.

---

<div class="post-metadata">

**Author:** ![cscherrer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cscherrer/32/7631_2.png) [@cscherrer](https://discourse.julialang.org/u/cscherrer)\
**Post date:** [December 13, 2017, 5:53pm UTC](https://discourse.julialang.org/t/reversediff-jl/764/8 "2017-12-13T17:53:32Z")

</div>

_“Since these tapes can be analyzed as computation graphs, my hope is that this infrastructure can eventually be rendered useful for non-AD purposes, such as performance optimization, scheduled parallel execution, and constraint programming.”_

Oh, that’s really interesting, @jrevels. A lot of probabilistic programming is in terms of “execution traces”, usually expressed in terms of coroutines or continuations. Computation of gradients is also really common in that context; maybe there’s a way to combine the two?

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [December 13, 2017, 6:36pm UTC](https://discourse.julialang.org/t/reversediff-jl/764/9 "2017-12-13T18:36:27Z")

</div>

@cscherrer that sounds a lot like what [Cassette.jl](https://github.com/jrevels/Cassette.jl/blob/master/docs/design.md) is aiming to do.

---

<div class="post-metadata">

**Author:** ![cscherrer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cscherrer/32/7631_2.png) [@cscherrer](https://discourse.julialang.org/u/cscherrer)\
**Post date:** [December 13, 2017, 9:18pm UTC](https://discourse.julialang.org/t/reversediff-jl/764/10 "2017-12-13T21:18:10Z")

</div>

Oh interesting, I hadn’t seen Cassette.