# AD in a SGD loop: one call to 'gradient' or many?

**URL:** <https://discourse.julialang.org/t/ad-in-a-sgd-loop-one-call-to-gradient-or-many/33793>\
**Category:** Machine Learning\
**Created:** [January 26, 2020, 1:45am UTC](https://discourse.julialang.org/t/ad-in-a-sgd-loop-one-call-to-gradient-or-many/33793 "2020-01-26T01:45:43Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![jaynick](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@jaynick](https://discourse.julialang.org/u/jaynick)\
**Post date:** [January 26, 2020, 1:45am UTC](https://discourse.julialang.org/t/ad-in-a-sgd-loop-one-call-to-gradient-or-many/33793/1 "2020-01-26T01:45:43Z")

</div>

A general question about how to think about automatic differentiation.

I might have thought the gradient is a **function** , that takes in a location and returns the up-hill vector at that location.

If an AD returned such a function, then one would obtain the function outside of the gradient descent loop, i.e. (pseudocode)

```
loss(x) = ...
dloss(x) = gradient(loss,x) # dloss() is constructed only once, outside the loop
x = randinit()
for iter = 1:n
    x = x - 0.01 * dloss(x)

```

However, if AD libaries only return a vector, not a function, then it is necessary to call the AD _inside_ the iteration.

Does anyone know which it is?

Honestly, I am finding flux and knet docs hard to understand in this aspect! It seems that the docs have either examples that are too high-level (using Adam and a SGD wrapper), too simple (not involving GD), or those that assume one knows how AD works internally. (I hope to, but not yet 🙂

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [January 26, 2020, 9:42am UTC](https://discourse.julialang.org/t/ad-in-a-sgd-loop-one-call-to-gradient-or-many/33793/2 "2020-01-26T09:42:26Z")

</div>

You can choose whatever you want. Either of

```julia
gradfun(x) = DiffLibrary.gradient(f,x)
for ...
    g = gradfun(x)
end

```

```julia
for ...
    g = DiffLibrary.gradient(f,x)
end

```

would work. You always have to call the AD in the loop, there’s no way around that. The AD is what calculates the gradient after all. Depending on the AD library, more or less of the gradient computations will be handled and optimized by the compiler. If you use a tape-based reverse ad a lot of things happens at runtime. If you use ForwardDiff or Zygote, a lot of things are optimized by the compiler. You still have to call the gradient function though.

---

<div class="post-metadata">

**Author:** ![jaynick](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@jaynick](https://discourse.julialang.org/u/jaynick)\
**Post date:** [January 26, 2020, 8:37pm UTC](https://discourse.julialang.org/t/ad-in-a-sgd-loop-one-call-to-gradient-or-many/33793/3 "2020-01-26T20:37:25Z")

</div>

Thank you.

From your answer I guess that constructing the **gradfun()** fucntion ie. this step

```julia
gradfun(x) = DiffLibrary.gradient(f,x)

```

is not so expensive compared to evaluating it, so there is no motivation to put it outside the loop?

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [January 26, 2020, 10:07pm UTC](https://discourse.julialang.org/t/ad-in-a-sgd-loop-one-call-to-gradient-or-many/33793/4 "2020-01-26T22:07:42Z")

</div>

No, nothing is done at compile time in one of the cases that cannot be done in the other. They should be exactly equivalent in performance

---

<div class="post-metadata">

**Author:** ![lineycroc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lineycroc/32/10078_2.png) [@lineycroc](https://discourse.julialang.org/u/lineycroc)\
**Post date:** [January 27, 2020, 2:37pm UTC](https://discourse.julialang.org/t/ad-in-a-sgd-loop-one-call-to-gradient-or-many/33793/5 "2020-01-27T14:37:51Z")

</div>

Not too familiar with the Julia implementations, but I found this document which seems to be a decent explanation of AD without being too detail-oriented too quickly if you’re interested (I particularly found the comparisons to numeric differentiation and symbolic differentiation helpful): [http://jmlr.org/papers/volume18/17-468/17-468.pdf](http://jmlr.org/papers/volume18/17-468/17-468.pdf).

Feel free to correct me, but my quick read of it is that:

- Automatic differentiation is numerically evaluating the derivative at a given point. It’s similar to symbolic differentiation in that it uses the chain rule, and may even be the same in some cases, but if there’s a branch in your code, AD would only take the relevant path for your input. You could write a function (outside a loop) that called AD, but every time you call that function in the loop, you are still doing the full evaluation of the derivative at the new point.
- If you want to create the gradient function outside the loop, I think you want symbolic differentiation, which would return a function that computes the derivative at any input point (where hopefully computing the function is much cheaper than finding it). The article (and wikipedia and a bunch of other sites) have a lot to say about how symbolic differentiation can be bad for complicated computer programs though.

---

<div class="post-metadata">

**Author:** ![jaynick](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@jaynick](https://discourse.julialang.org/u/jaynick)\
**Post date:** [January 27, 2020, 6:09pm UTC](https://discourse.julialang.org/t/ad-in-a-sgd-loop-one-call-to-gradient-or-many/33793/6 "2020-01-27T18:09:21Z")

</div>

Thank you, helpful comments.

Especially, I now understand that the gradient “function” cannot be constructed outside the loop, if it contains branching.