[ForwardDiff Question]: Jacobian of a function that calls another function

General Usage
,
1

I don’t understand why my vector-valued function does not produce Jacobian using ForwardDiff.jl. Could you help me how can I make it work? Ultimately I am gonna call that resulting Jacobian P and try to compute P’P\P(some vector). Below is the truncated work of mine that keeps what I intend to do.

using ForwardDiff, Parameters

@with_kw struct Primitives
    β::Float64 = 0.96 # discount rate
end

function u(θ; M,A,E,N,B,W,j)
    δ_1, δ_2 = θ
    u = (δ_1^2*A) + δ_2^2*A^2+j
    return u
end

function Z(θ)
    Z1 = zeros(3)
    β = 0.96
    for a=2:4
        Z1[a-1] = u(θ,M=0,A=a+0,E=1,N=0,B=0,W=0,j=1) + β*u(θ,M=0,A=a+0,E=1,N=0,B=0,W=0,j=2))
    end
    return Z1
end
Z([1.0,1.0]) # this yields correct Vector{Float64}
θ_init = [1.0,1.0] # Vector{Float64}
ForwardDiff.jacobian(θ->Z(θ), θ_init) # this is where I get an error

I get the following error message:
MethodError: no method matching Float64(::ForwardDiff.Dual{ForwardDiff.Tag{var"#30#31",Float64},Float64,2}}

I am certain that my function Z accepts Vector{Float64} as an argument and its output is also a Vector{Float64}. For function u(), it also accepts Vector{Float64} as an argument.

Thank you for your input :slight_smile:

2

ForwardDiff works by running the function with its own Dual number types. The error message tells you that your function is trying to convert one of these to Float64, which won’t work (as it would have to throw away the information you are looking for). You will generally need to make things more generic to allow this, e.g. struct Primitives{T} β::T end and also zeros(typeof(θ), 3) (which is a good idea anyway, to allow e.g. Float32).

Also, please check that your code runs (at least until the error in question!) on a fresh session of Julia, and write ``` before & after to quote it.

4 Likes
3

Thank you for your reply. I have modified the question according to your suggestion. The above script is all I have and I verified that after your suggestion, I have the error message even before ForwardDiff.jacobian:

using Parameters, ForwardDiff
@with_kw struct Primitives{T}
    β::T # discount rate
end

function u(θ; M,A,E,N,B,W,j)
    δ_1, δ_2 = θ
    u = (δ_1^2*A) + δ_2^2*A^2+j
    return u
end

function Z(θ)
    Z1 = zeros(typeof(θ), 3)
    @unpack β = Primitives()
    for a=22:24
        Z1[a-21] = (u(θ,M=0,A=a+0,E=1,N=0,B=0,W=0,j=1)+β*u(θ,M=0,A=a+0,E=1,N=0,B=0,W=0,j=2))
    end
    return Z1
end
Z([1.0,1.0])
θ_init = [1.0,1.0]
ForwardDiff.jacobian(θ->Z(θ), θ_init)

Z([1.0,1.0])

returns

MethodError: no method matching zero(::Type{Array{Float64,1}})
Closest candidates are:
  zero(!Matched::Type{Missing}) at missing.jl:103
  zero(!Matched::Type{LibGit2.GitHash}) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\LibGit2\src\oid.jl:220
  zero(!Matched::Type{Pkg.Resolve.VersionWeight}) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Resolve\versionweights.jl:15
  ...
zeros(::Type{Array{Float64,1}}, ::Tuple{Int64}) at array.jl:505
zeros(::Type{Array{Float64,1}}, ::Int64) at array.jl:500
Z(::Array{Float64,1}) at experimentHouse.jl:38
top-level scope at experimentHouse.jl:45

Also, ForwardDiff.jacobian(θ->Z(θ), θ_init) returnst the following:

MethodError: no method matching zero(::Type{Array{ForwardDiff.Dual{ForwardDiff.Tag{var"#21#22",Float64},Float64,2},1}})
Closest candidates are:
  zero(!Matched::Type{Missing}) at missing.jl:103
  zero(!Matched::Type{LibGit2.GitHash}) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\LibGit2\src\oid.jl:220
  zero(!Matched::Type{Pkg.Resolve.VersionWeight}) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Resolve\versionweights.jl:15
  ...
zeros(::Type{Array{ForwardDiff.Dual{ForwardDiff.Tag{var"#21#22",Float64},Float64,2},1}}, ::Tuple{Int64}) at array.jl:505
zeros(::Type{Array{ForwardDiff.Dual{ForwardDiff.Tag{var"#21#22",Float64},Float64,2},1}}, ::Int64) at array.jl:500
Z(::Array{ForwardDiff.Dual{ForwardDiff.Tag{var"#21#22",Float64},Float64,2},1}) at experimentHouse.jl:38
(::var"#21#22")(::Array{ForwardDiff.Dual{ForwardDiff.Tag{var"#21#22",Float64},Float64,2},1}) at experimentHouse.jl:48
vector_mode_dual_eval(::var"#21#22", ::Array{Float64,1}, ::ForwardDiff.JacobianConfig{ForwardDiff.Tag{var"#21#22",Float64},Float64,2,Array{ForwardDiff.Dual{ForwardDiff.Tag{var"#21#22",Float64},Float64,2},1}}) at apiutils.jl:37
vector_mode_jacobian(::var"#21#22", ::Array{Float64,1}, ::ForwardDiff.JacobianConfig{ForwardDiff.Tag{var"#21#22",Float64},Float64,2,Array{ForwardDiff.Dual{ForwardDiff.Tag{var"#21#22",Float64},Float64,2},1}}) at jacobian.jl:140
jacobian(::Function, ::Array{Float64,1}, ::ForwardDiff.JacobianConfig{ForwardDiff.Tag{var"#21#22",Float64},Float64,2,Array{ForwardDiff.Dual{ForwardDiff.Tag{var"#21#22",Float64},Float64,2},1}}, ::Val{true}) at jacobian.jl:17
jacobian(::Function, ::Array{Float64,1}, ::ForwardDiff.JacobianConfig{ForwardDiff.Tag{var"#21#22",Float64},Float64,2,Array{ForwardDiff.Dual{ForwardDiff.Tag{var"#21#22",Float64},Float64,2},1}}) at jacobian.jl:15
jacobian(::Function, ::Array{Float64,1}) at jacobian.jl:15
top-level scope at experimentHouse.jl:48

As I am new to the Dual number types, I am more lost here… why do we use write curly brackets in the end of Primitives{T}? Does it free \beta to change its types when needed?

4

Change typeof(x) for eltype(x)

3 Likes
5

Sorry my mistake re typeof / eltype. And on reading a bit more carefully, the original Primitives might be fine too.

Yes, this would allow Primitives to store β of any type, although the type (and value, since it’s not a mutable struct) are fixed for the life of that object. Making T part of the wrapper’s type means there’s no need for the compiler to allow for surprises upon unwrapping. But this is only essential if it’s on the path from θ_init to Z1, i.e. if you created one with input from θ.

3 Likes
6

Thank you so much for your reply. It works now :slight_smile:

1 Like
7

Thanks for the clarification!

1 Like