Evaluation, gradient and Hessian of a scalar function for multiple values using ForwardDiff.jl

Specific Domains Numerics
1

Hello,

I would like to evaluate a scalar-function f: R \xrightarrow{} R as well as its derivative and hessian at around 200 values using ForwardDiff.jl

To speed-up the computations, is it possible to precompute an object like the graph of the function and apply it to the different values? From my understanding, it would be a waste of time to recompute the same thing multiple times.

2

An equivalent to what you’re talking about already happens automagically. ForwardDiff works by calling your code with a type (dual) that causes your code to also compute derivatives. As such your code will run with a new type, so compilation occurs to optimize your code.

3

Thank you for your answer. Ok, so the following code will use the optimized code for f?

using ForwardDiff
using BenchmarkTools

function timing()
x = randn(200)
g(x) = (x^4-5x^3*x^2+1)*exp(-x^2/2)
@btime ForwardDiff.derivative.(g, x)
end

How do I get the evaluation and the hessian of the function at the same time?

To compute the second derivative of a scalar function, is there a cleaner way than using nested ForwardDiff.derivative calls ?

dg = zeros(200)
map(xi->ForwardDiff.derivative(z->ForwardDiff.derivative(g, z), xi),x)
4

Check DiffResults.jl, sounds like is what you want

5

The problem is that DiffResults.jl computes simultaneously f(x), \nabla f(x), H(f(x)) but solely for the same vector, not for repeated use of the same function f for different values.

6

Calculating at a different point requires everything to be recalculated. As @Oscar_Smith said, once you call ForwardDiff once, the code should have been compiled for your particular function, and the compiled version will be used for all the evaluations.