Evaluation, gradient and Hessian of a scalar function for multiple values using ForwardDiff.jl


I would like to evaluate a scalar-function f: R \xrightarrow{} R as well as its derivative and hessian at around 200 values using ForwardDiff.jl

To speed-up the computations, is it possible to precompute an object like the graph of the function and apply it to the different values? From my understanding, it would be a waste of time to recompute the same thing multiple times.

An equivalent to what you’re talking about already happens automagically. ForwardDiff works by calling your code with a type (dual) that causes your code to also compute derivatives. As such your code will run with a new type, so compilation occurs to optimize your code.


Thank you for your answer. Ok, so the following code will use the optimized code for f?

using ForwardDiff
using BenchmarkTools

function timing()
x = randn(200)
g(x) = (x^4-5x^3*x^2+1)*exp(-x^2/2)
@btime ForwardDiff.derivative.(g, x)

How do I get the evaluation and the hessian of the function at the same time?

To compute the second derivative of a scalar function, is there a cleaner way than using nested ForwardDiff.derivative calls ?

dg = zeros(200)
map(xi->ForwardDiff.derivative(z->ForwardDiff.derivative(g, z), xi),x)

Check DiffResults.jl, sounds like is what you want

1 Like

The problem is that DiffResults.jl computes simultaneously f(x), \nabla f(x), H(f(x)) but solely for the same vector, not for repeated use of the same function f for different values.

Calculating at a different point requires everything to be recalculated. As @Oscar_Smith said, once you call ForwardDiff once, the code should have been compiled for your particular function, and the compiled version will be used for all the evaluations.