Essentially, TensorFlow provides 3 main advantages:

- Automated differentiation.
- Code generation for CPU and GPU.
- Distributed computations.

I donâ€™t know much about TFâ€™s model of distributed computations, so canâ€™t really comment on this.

I wrote specifically *automated* differentiation because in TF itâ€™s not exactly the same as *automatic* differentiation e.g. in Knet.jl. Citing @denizyuret:

Automatic differentiation is the idea of using symbolic derivatives only at the level of elementary operations, and computing the gradient of a compound function by applying the chain rule to intermediate numerical results. For example, pure symbolic differentiation of \sin^2(x) could give us 2\sin(x)\cos(x) directly. Automatic differentiation would use the intermediate numerical values x_1=\sin(x), x_2=x_1^2 and the elementary derivatives dx_2/dx_1=2x_1, dx_1/dx=\cos(x) to compute the same answer without ever building a full gradient expression.

AD is pretty good, actually, especially being backed by GPU arrays. Yet, as you mentioned, it doesnâ€™t create a computational graph which limits many optimizations.

An alternative approach is to use *symbolic* differentiation. SD is less straightforward to implement and has its own limitations (e.g. no loops on loss function), but it can produce exactly what AD is missing - computational graph (for which we already have Juliaâ€™s AST). To my knowledge, there are currently 2 packages providing symbolic differentiation on array types - ReverseDiffSource.jl by FrĂ©dĂ©ric Testard and mine XDiff.jl. Both are not in the best shape (ReverseDiffSource doesnâ€™t support Julia 0.6 yet, XDiff.jl is currently under the major refactoring), but if you are looking for symbolic computational graphs like in TensorFlow helping one of these projects may be a good start.

Code generation comes from symbolic graphs and shouldnâ€™t be too hard (especially given awesome CUDANative.jl), yet making it produce really highly optimized code may take many man-hours, and this is exactly where TF has the advantage over not-so-well-known projects.

I can dive deeper into the details of (1) and (2) if you really want to step this way, but you should be aware that this way is quite long yet.