Hello, I am trying to compute the Hessian vector product of a Flux model (with some vector) on the GPU. This is giving me some errors, which I think are realted specifically to ForwardDiff. Any help would be greatly appreciated! The following defines an Hvp operator which works fine on the CPU: u…

Hessian Vector Products on GPU using ForwardDiff, Zygote, and Flux

ChrisRackauckas January 30, 2022, 5:03pm 10

I haven’t dug into the details of your case, but it certainly sounds like that’s the right track. I’m not sure it’s using the fused sigmoid rule, but it could be, in which case that would need an frule.

Topic		Replies	Views
How to calculate the Hessian vector product in a Flux model? Machine Learning question , gpu , flux , zygote , forwarddiff	1	313	June 13, 2023
Hessian Vector products in Flux with Zygote Machine Learning flux , zygote	4	1069	December 22, 2021
Hessian inside a Flux loss function Machine Learning question	3	809	February 26, 2021
(yet another) Zygote: mutating arrays is not supported Machine Learning	10	1551	July 7, 2021
ForwardDiff of function with internal derivatives using CuArrays GPU forwarddiff	8	923	March 31, 2021

Hessian Vector Products on GPU using ForwardDiff, Zygote, and Flux

Related topics