ForwardDiff.derivative pointwise

General Usage Performance
1

Hello,
is it possible to use ForwardDiff in the following way. I have a function (a forward diff through a neural network). I want to take the second derivative of the output with respect to the input.
If I use n input points, the n outputs are point independent.

Is it possible to use ForwardDiff for this? (The Jacobian obviously takes too many derivatives).

Thank you very much for your time.

2

Hi I wrote an own small extension for the forwardDiff api.

Maby someone will need this too:

@inline function derivative(f::F, x::AbstractArray{R}) where {F,R<:Real}
    T = typeof(ForwardDiff.Tag(f, R))
    
    dual_vec = ForwardDiff.Dual{T}.(x, ones(size(x)))

    return ForwardDiff.extract_derivative(T,f(dual_vec))
end
3

Zygote.jl cant take the derivative of a loss function containing the above code.

MethodError: no method matching Zygote.OneElement(::NamedTuple{(:value, :partials), Tuple{Nothing, Zygote.OneElement{Float64, 1, Tuple{Int64}, Tuple{Base.OneTo{Int64}}}}}, ::Tuple{Int64}, 
::Tuple{Base.OneTo{Int64}})
Closest candidates are:
  Zygote.OneElement(::T, ::I, ::A) where {N, T<:Number, I<:Tuple{Vararg{Int64, N}}, A<:Tuple{Vararg{AbstractUnitRange, N}}} at C:\Users\Lenovo\.julia\packages\Zygote\arfUC\src\lib\array.jl:69

There is a problem with the namedTuple of the Partials type I guess. Is there a work around?