Cf
https://github.com/JuliaDiff/ForwardDiff.jl/pull/315
Something similar for Hessian may help resolve this.