Hello,
I would like to compute the Jacobian and the Hessian of a function f: \mathbb{R}^n \xrightarrow{} \mathbb{R}^m with n,m \sim60. This function involves multiple Fourier transforms and nonlinear transformations that make it difficult to derive by hand the Jacobian and Hessian. The Jacobian and Hessian are not sparse =(
I am now using FiniteDiff.jl but getting the Jacobian takes 300ms (Forward differentiation) while a function evaluation takes 2ms. I need to evaluate this Jacobian for 300 input vectors. How can I speed up these computations? I don’t need machine precision results, an error of 10^{-6} is good enough. Can we get at the same time the Jacobian and the Hessian?
I can’t apply Zygote because of the FFT transforms
Calculating the jacobian and hessian are expensive operations. Why so you expect to be able to speed them up?
What do you need to do with them? Eg if you immediately multiply the jacobian by a vector it should be possible to do that more efficiently.
Thank you for your answer,
There are sometime tricks that I am not aware of, just asking for ideas =)
With finite differences, I need to evaluate the same function a lot of time, maybe it is worth trying to approximate the function first…
I am looking at a sensitivity analysis problem, and I have to compute terms like h^T h with h = \partial_{x_i} \partial_{x_j} f(x) and (\partial_{x_i} f_{k}(x))^2, for i, j = 1,\ldots,n and k=1, \ldots, m
1 Like
Why wouldn’t Zygote work with the FFTs? Zygote can differentiate at least over FFTW
The package that evaluates the function f is PotentialFlow.jl(https://github.com/darwindarak/PotentialFlow.jl) that has been written in-place and Zygote doesn’t support mutating arrays.
I might need to rewrite part of it to use Zygote.
Also the cost of getting one component of f is the same as getting all the components, how can I modify FiniteDiff.jl to compute the hessian for all the components of f at the same time?
.
This seems like a use case for ForwardDiff + DiffResults
3 Likes