# ANN: XDiff.jl - an expression differentiation package

**URL:** <https://discourse.julialang.org/t/ann-xdiff-jl-an-expression-differentiation-package/1642>\
**Category:** Machine Learning\
**Created:** [January 22, 2017, 6:47pm UTC](https://discourse.julialang.org/t/ann-xdiff-jl-an-expression-differentiation-package/1642 "2017-01-22T18:47:27Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![dfdx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dfdx/32/120_2.png) [@dfdx](https://discourse.julialang.org/u/dfdx)\
**Post date:** [January 22, 2017, 10:52pm UTC](https://discourse.julialang.org/t/ann-xdiff-jl-an-expression-differentiation-package/1642/4 "2017-01-22T22:52:33Z")

</div>

XDiff.jl takes roots in ReverseDiffSource.jl, hence similar names and semantics. However, there are several important differences:

1. ReverseDiffSource [can’t handle nested functions](https://github.com/JuliaDiff/ReverseDiffSource.jl/issues/38). While XDiff isn’t perfect in this sense too, it can handle all scalar and most element-wise and broadcasting functions. For example, there’s derivative only for `log(x)` (single argument) defined in the package, but if you call `rdiff` on `log(b, x)` (two arguments), it will extract code for this method, infer a new differentiation rule for it and add to cache. In practice, this means that you don’t need to define your cost functions as a single long function but can easily decompose it into a number of smaller ones as you would do in normal code.

2. XDiff supports differentiation of vector-valued functions. In the context of ML, it’s mostly valuable for analysis only since cost functions are normally scalar-valued. But I’ve already encountered a couple of use cases (e.g. in finance) where both - input and output of a function - are vectors.

3. The final step of differentiation is code generation. Currently, there are only 2 formats available - vectorized and Einstein notation - but the final goal is to add pluggable code generators. Imagine that with a single argument you can produce vectorized code or code with fused loops, or BLAS-optimized, or code for GPU, etc. Though, this is mostly part of [Espresso.jl](https://discourse.julialang.org/t/ann-espresso-jl-an-expression-transformation-package/1641) vision.

---

_[View the full topic](https://discourse.julialang.org/t/ann-xdiff-jl-an-expression-differentiation-package/1642)._