Automatic Differentiation of arbitrary-order derivatives

Specific Domains Numerics
1

I was wondering if it is possible to programmatically do higher-order automatic differentiation using ForwardDiff (or ReverseDiff).

Here’s a MWE that shows a non-optimal way to get specific derivatives (hard-coded):

import ForwardDiff

function myFun(degree,variate)
    f = (variate^2 - 1)^degree
    return f
end

# Pick a degree
degree = 4

# Make functions for arbitrarily high derivatives
D1(x) = ForwardDiff.derivative(x->myFun(degree,x),x)
D2(x) = ForwardDiff.derivative(x->ForwardDiff.derivative(x->myFun(degree,x),x),x)
D3(x) = ForwardDiff.derivative(x->ForwardDiff.derivative(x->ForwardDiff.derivative(x->myFun(degree,x),x),x),x)
D4(x) = ForwardDiff.derivative(x->ForwardDiff.derivative(x->ForwardDiff.derivative(x->ForwardDiff.derivative(x->myFun(degree,x),x),x),x),x)
# etc

## Evaluate the derivatives at a point
v1 = D1(sqrt(2)/4)
v2 = D2(sqrt(2)/4)
v3 = D3(sqrt(2)/4)
v4 = D4(sqrt(2)/4)
# etc

But I would like something that provides the same functionality / user-experience as these MATLAB MWEs:

  1. First Matlab Example
clear
x = sym('x','real');
degree = sym('degree',{'positive','integer'});

nDeriv = 4; % I want the 4th derivative in this case, but this could be...
% nDeriv = randi([0 100])
f(x,degree) = (x^2 - 1) ^ degree; % Define f(x) as symbolic function
df = diff(f,x,nDeriv); % Construct a function that evaluates the value of the 4th derivative at a point

deg = 4;
df_val = df(sqrt(2)/4,deg) % Compute the 4th derivative at a point - is symbolic
df_val = double(df_val) % Cast to a double
  1. Second Matlab Example
clear
x = sym('x','real');
degree = sym('degree',{'positive','integer'});

nDeriv = 4; % I want the 4th derivative in this case, but this could be...
% nDeriv = randi([0 100])
f(x,degree) = (x^2 - 1) ^ degree; % Define f(x) as symbolic function
for n = 0:nDeriv
    if n == 0
        df = f;
    else
        df = diff(df,x); % Construct a function that evaluates the value of the 4th derivative at a point
    end
end

deg = 4;
df_val = df(sqrt(2)/4,deg) % Compute the 4th derivative at a point - is symbolic
df_val = double(df_val) % Cast to a double

I’ve tried doing a Julia code that follows the idea of the second Matlab example

import ForwardDiff

function myFun(degree,variate)
    f = (variate^2 - 1)^degree
    return f
end

degree = 4
nDeriv = 4
df(x) = x->myFun(degree,x)
for n = 1:nDeriv
    df(x) = ForwardDiff.derivative(x->df(x),x)
end

But evaluating df(sqrt(2)/4) results in something like #23 (generic function with 1 method) instead of returning a double (should be -9.750000000000007)

2

Try making df an anonymous function instead of a named function.

3

I really like this snippet from Roots.jl docs

import ForwardDiff
degree = 4 
myFun(d,x)=(x^2-1)^d
D(f) = x -> ForwardDiff.derivative(f, float(x))
D(f, n) = n > 1 ? D(D(f),n-1) : D(f)
f = x-> myFun(degree, x) 
D(f, 6)(2) #sixth order scalar derivative
7 Likes
4

I’m not sure whether this answers your question, but the following package may be of interest:

julia> using TaylorSeries

julia> t = Taylor1(Float64, 6)
 1.0 t + 𝒪(t⁷)

julia> sqrt(1 + t)/2
 0.5 + 0.25 t - 0.0625 t² + 0.03125 t³ - 0.01953125 t⁴ + 0.013671875 t⁵ - 0.01025390625 t⁶ + 𝒪(t⁷)

julia> myFun(4, 2+t) # longemen3000's function
 81.0 + 432.0 t + 972.0 t² + 1200.0 t³ + 886.0 t⁴ + 400.0 t⁵ + 108.0 t⁶ + 𝒪(t⁷)

julia> 77760 / factorial(6)
108.0
3 Likes