# Comparison of automatic differentiation tools from 2016 still accurate?

**URL:** <https://discourse.julialang.org/t/comparison-of-automatic-differentiation-tools-from-2016-still-accurate/8341>\
**Category:** Numerics\
**Tags:** differentiation\
**Created:** [January 13, 2018, 2:38pm UTC](https://discourse.julialang.org/t/comparison-of-automatic-differentiation-tools-from-2016-still-accurate/8341 "2018-01-13T14:38:33Z")\
**Posts on this page:** 2\
**Page:** 3

<div class="post-metadata">

**Author:** ![MikeInnes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikeinnes/32/3656_2.png) [@MikeInnes](https://discourse.julialang.org/u/MikeInnes)\
**Post date:** [August 16, 2018, 2:20pm UTC](https://discourse.julialang.org/t/comparison-of-automatic-differentiation-tools-from-2016-still-accurate/8341/41 "2018-08-16T14:20:08Z")

</div>

I’m not sure what you mean by “breaking SSA” – this kind of loop would be represented using phi nodes (though supporting mutable slots is actually fine too). If you want an intuition for how control flow like this might be handled, [tangent’s docs](https://github.com/google/tangent#control-flow) do a good job.

---

<div class="post-metadata">

**Author:** ![dfdx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dfdx/32/120_2.png) [@dfdx](https://discourse.julialang.org/u/dfdx)\
**Post date:** [August 16, 2018, 6:18pm UTC](https://discourse.julialang.org/t/comparison-of-automatic-differentiation-tools-from-2016-still-accurate/8341/42 "2018-08-16T18:18:50Z")

</div>

Oh, I didn’t know “SSA-form” refers to a specific technique in compiler optimizations, now I see your point. Indeed, a compiler handling inputs, outputs and variable assignments greatly simplifies the task. Yet, some bookkeeping is still needed.

I was referring to “SSA” in a sense of mathematical variable that can’t be modified or overwritten. In general, if you have a recurrence x\_i = f(x\_{i-1}), you need to keep all the x\_k to backpropagate through them. Tangent does it by pushing values of re-assigned variable onto a stack, which I personally dislike since it breaks further optimizations based on dependency graph.

In Julia SSA-form, as far as I understand, the mutable part of a loop is hidden inside \phi-node. E.g. in:

```julia
function foo(x)
    while x < 100
        x = x * 2
    end
    return x
end

@code_typed foo(4) # is it the right way to get SSA form? 

```

which generates:

```julia
  1 ─ nothing::Nothing │ 
2 2 ┄ %2 = φ (#1 => _2, #3 => %5)::Int64 │ 
  │ %3 = (Base.slt_int)(%2, 100)::Bool │╻ <
  └── goto #4 if not %3 │ 
3 3 ─ %5 = (Base.mul_int)(%2, 2)::Int64 │╻ *
  └── goto #2 │ 
5 4 ─ return %2

```

I understand that this is slot `_2` inside `φ (#1 => _2, #3 => %5)` that accumulates changes, right? If so, you need to keep track of its values.

There also should be a variable to count loop iterations and, if we take ML applications into account, the generated code should be GPU-friendly, and some other complications, but it would be very interesting to see it implemented in Zygote.jl!

[Previous page](https://discourse.julialang.org/t/comparison-of-automatic-differentiation-tools-from-2016-still-accurate/8341.md?page=2)