Automatic differentiation in different cases

Specific Domains Numerics
, ,
1

I am trying to figure out :

  1. how to automatically differentiate some function f(x1,x2,x3…) for vector of numbers. I want to use one package preferably , but if I have to mix, I want to mix zygote with ForwardDiff.

I think I have this down in using something like :

g(x,y)=x^2+y^2
ForwardDiff.gradient(z → g(z[1], z[2]), [1.0, 2.0])

I guess if I wanted to take gradient for a vector, where I evaluate each partial derivative at the same point I could do :

X=[1.0,2.0,3.0]

for elem in X
ForwardDiff.gradient(z ->g(z[1], z[2]),[elem,elem])
end

This seems hacky and inefficient , and I know I can use static vectors to speed things up somewhat.

Next:

  1. I want to evaluate a laplacian , a double gradient, for some multivariable function f(x1,x2,x3…) for a vector of numbers.

I guess I could chain the gradient function?

  1. I want to evaluate a gradient f(x1,x2,x3,…) where I only evaluate for one variable, and I set say x2=1.0, x3=3.0

The way I have been doing this is grotesque , I simply recompile my function as a single variable function with a new global variable value put in for the x2,x3…

Thank you! I feel that this thread should be very useful.

2

I didn’t fully understand all those questions, but here is some code for ForwardDiff that can hopefully be helpful. Generally, I’d say it’s easier to work with functions f(x) where x is a vector, rather than functions f(x1,x2,x3,...) when using ForwardDiff. To make your future posts easier to read, include code in backticks ` (single for inline, triple for multiline).

using ForwardDiff
using StaticArrays
## --

# Function
f(x1,x2) = x1^2 + x2^2
f(x) = f(x[1],x[2])

# Gradient
∇f(x) = ForwardDiff.gradient( f , x )
∇f(x1,x2) = ∇f(SVector(x1,x2))

# Double gradient (?)
∇²f(x) = ForwardDiff.hessian( f , x )
∇²f(x1,x2) = ∇²f(SVector(x1,x2))

# Laplacian 
f1(x) = ForwardDiff.derivative( x1 -> f(x1,x[2]) , x[1] )
f2(x) = ForwardDiff.derivative( x2 -> f(x[1],x2) , x[2] )
f11(x) = ForwardDiff.derivative( x1 -> f1( SVector(x1,x[2]) ) , x[1] )
f22(x) = ForwardDiff.derivative( x2 -> f2( SVector(x[1],x2) ) , x[2] )
Δf(x) = f11(x) + f22(x) 
Δf(x1,x2) = Δf( SVector(x1,x2) )

# Evaluate

x1 = 1.0
x2 = 2.0

f(x1,x2)
∇f(x1,x2)
∇²f(x1,x2)
Δf(x1,x2)

# For a vector of numbers?

num_vec = [ SVector(rand(),rand()) for _ ∈ 1:5 ]

Δf.(num_vec)

The code for the Laplacian is admittedly a little gnarly, if you want to do many higher-order directional derivatives, then TaylorDiff.jl is probably a better choice.

3

Laplacian is the trace of Hessian.

import LinearAlgebra:tr
g(x) = x[1]^2*sin(x[2]-x[1])
Δg=tr(ForwardDiff.hessian(g, [π/2, π/6]))
1 Like