In a calculation involving an f: R^n \to R^m function, where n ranges from 50 to a few hundred and m \ge n with a similar scale, I have the option of either calculating the Jacobian J once and saving it, or calculating Jv, Jw, uJ for three different vectors v, w, u. I can (and will) benchma…

First of all, a few questions: Is the Jacobian sparse? How many times are you performing each operation? Does everything vary at each time, or are you in a setting where you have a single x, which determines J_f(x), and many different triplets (u, v, w)? Say your function f costs \tau to compute …

How to think about the cost of vJ and Jv (pullback and pushforward)

Specific Domains Numerics

Tamas_Papp May 15, 2025, 8:57am 5

Yes, it has mutation and some allocations too. It is type stable and works with ForwardDiff.Dual.

Topic		Replies	Views
Automatic Differentiation Slow (Slower than Finite-Differences) Optimization (Mathematical)	18	3557	May 31, 2018
Why can forwarddiff get gradient wrt. to multiple parameters in 1 function call? Numerics question	5	1632	September 3, 2017
Efficient pushforward (Jacobian vector product) with multiple vectors using automatic differentiation Performance autodiff	13	606	May 27, 2023
Improve speed of vector-jacobian product Performance reversediff , autodiff	2	527	November 16, 2023
Simultaneous Jacobian and Hessian computations Numerics question , zygote , finitediff	7	816	August 13, 2020

How to think about the cost of vJ and Jv (pullback and pushforward)

Related topics