# What is the right way to compute Gradient of function of 2 variables with Auto Differentiation?

**URL:** <https://discourse.julialang.org/t/what-is-the-right-way-to-compute-gradient-of-function-of-2-variables-with-auto-differentiation/15138>\
**Category:** New to Julia\
**Tags:** package, gradient\
**Created:** [September 18, 2018, 5:51pm UTC](https://discourse.julialang.org/t/what-is-the-right-way-to-compute-gradient-of-function-of-2-variables-with-auto-differentiation/15138 "2018-09-18T17:51:26Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Andrey.Borzunov](https://avatars.discourse-cdn.com/v4/letter/a/9f8e36/32.png) [@Andrey.Borzunov](https://discourse.julialang.org/u/Andrey.Borzunov)\
**Post date:** [September 18, 2018, 5:51pm UTC](https://discourse.julialang.org/t/what-is-the-right-way-to-compute-gradient-of-function-of-2-variables-with-auto-differentiation/15138/1 "2018-09-18T17:51:26Z")

</div>

I’m terribly confused with number of packages that provide autodiff functionalities and it’s peculiarity.

I’m required to compute gradient of multivariable function (e.g. `f(x,y)`, where `x,y` are Numbers).  
I found that AutoDiffSource and ReverseDiffSource are not supported.

I succeeded to do it with ReverseDiff.jl package in the next way:

```julia
f(a, b) = (1-a).^2 + 100*(a-b.^2).^2;
using ReverseDiff
g = (x,y) -> ReverseDiff.gradient(f, (x,y))
g([1],[1])

```

But it looks queer to me.

I have to pass explicit Arrays, and write `.*` operations in function definition. Why?

I succeeded to compute it with ForwardDiff.jl with derivative but not gradient.

```julia
g(a, b) = (1-a)^2 + 100*(a-b^2)^2
dfdx = x -> ForwardDiff.derivative(y -> g(x,y), x)
dfdy = y -> ForwardDiff.derivative(x -> g(x,y), y)

@show dfdx(1)
@show dfdy(2)

```

Is this two solutions are equivalent?

What is the right way to solve this problem?

Thx!

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [September 18, 2018, 7:24pm UTC](https://discourse.julialang.org/t/what-is-the-right-way-to-compute-gradient-of-function-of-2-variables-with-auto-differentiation/15138/2 "2018-09-18T19:24:14Z")

</div>

What derivatives do you want? Do you mean:

```julia
julia> g(a, b) = (1-a)^2 + 100*(a-b^2)^2
g (generic function with 1 method)

julia> ForwardDiff.gradient(z -> g(z[1], z[2]), [1.0, 2.0])
2-element Array{Float64,1}:
 -600.0
 2400.0

```

Giving you `[df/da, df/db]` at `a = 1.0, b = 2.0`

---

<div class="post-metadata">

**Author:** ![Andrey.Borzunov](https://avatars.discourse-cdn.com/v4/letter/a/9f8e36/32.png) [@Andrey.Borzunov](https://discourse.julialang.org/u/Andrey.Borzunov)\
**Post date:** [September 18, 2018, 7:37pm UTC](https://discourse.julialang.org/t/what-is-the-right-way-to-compute-gradient-of-function-of-2-variables-with-auto-differentiation/15138/3 "2018-09-18T19:37:44Z")

</div>

I want `grad_f(x,y)` which can evaluate gradient at arbitrary point `(x,y)`.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [September 18, 2018, 7:43pm UTC](https://discourse.julialang.org/t/what-is-the-right-way-to-compute-gradient-of-function-of-2-variables-with-auto-differentiation/15138/4 "2018-09-18T19:43:39Z")

</div>

```julia
julia> grad(x, y) = ForwardDiff.gradient(z -> g(z[1], z[2]), [x, y])
grad (generic function with 1 method)

julia> grad(2,3)
2-element Array{Int64,1}:
 -1398
  8400

```

Points are typically defined in Julia using Vectors (or for small dimensions StaticVectors). So writing it something like

```julia
julia> g(x) = (1-x[1])^2 + 100*(x[1]-x[2]^2)^2 # could of course unpack x into a and b here
g (generic function with 2 methods)

julia> grad(x) = ForwardDiff.gradient(g, x)
grad (generic function with 2 methods)

julia> grad([2,3])
2-element Array{Int64,1}:
 -1398
  8400

julia> using StaticArrays # good for small dim vectors

julia> grad(SVector(2,3))
2-element SArray{Tuple{2},Int64,1,2}:
 -1398
  8400

```

might be a bit more natural.

Also, note:

```julia
julia> using BenchmarkTools

julia> @btime grad($([2,3]));
  4.436 μs (4 allocations: 304 bytes)

julia> @btime grad($(SVector(2,3)));
  2.089 ns (0 allocations: 0 bytes)

```

which is a 2000x perf difference (which of course is due to how simple g is in this case)

---

<div class="post-metadata">

**Author:** ![Andrey.Borzunov](https://avatars.discourse-cdn.com/v4/letter/a/9f8e36/32.png) [@Andrey.Borzunov](https://discourse.julialang.org/u/Andrey.Borzunov)\
**Post date:** [September 18, 2018, 7:54pm UTC](https://discourse.julialang.org/t/what-is-the-right-way-to-compute-gradient-of-function-of-2-variables-with-auto-differentiation/15138/5 "2018-09-18T19:54:44Z")

</div>

Ok, looks like this is what I’m looking for.

Now, I’m chasing only convenient syntax for me(I will abandon this habit).

We can’t pass tuple to ForwardDiff like this

```julia
∇g(x,y) = ForwardDiff.gradient((x,y) -> g(z[1], z[2]), (x,y))

```

hence we can only use syntax where points are represented as vectors, not tuples.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [September 18, 2018, 7:59pm UTC](https://discourse.julialang.org/t/what-is-the-right-way-to-compute-gradient-of-function-of-2-variables-with-auto-differentiation/15138/6 "2018-09-18T19:59:15Z")

</div>

Yes, for tuples you would convert it to an Array (or for better performance, StaticArray)

```julia
julia> ∇g(x,y) = ForwardDiff.gradient(z -> g(z[1], z[2]), SVector(x,y))
∇g (generic function with 1 method)

julia> ∇g(1,2)
2-element SArray{Tuple{2},Int64,1,2}:
 -600
 2400

```

---

<div class="post-metadata">

**Author:** ![Andrey.Borzunov](https://avatars.discourse-cdn.com/v4/letter/a/9f8e36/32.png) [@Andrey.Borzunov](https://discourse.julialang.org/u/Andrey.Borzunov)\
**Post date:** [September 18, 2018, 8:03pm UTC](https://discourse.julialang.org/t/what-is-the-right-way-to-compute-gradient-of-function-of-2-variables-with-auto-differentiation/15138/7 "2018-09-18T20:03:42Z")

</div>

Oh, perfect. Thank you a lot.