Let’s say I have a function `f`

over an input `x`

and a set of parameters `w`

. For example:

```
f = (x, w) -> sum(x .* w)
```

I would like to compute the cross-terms of the hessian of `f`

. That is, I would like to compute the gradient of `g`

with respect to `w`

where `g`

itself is the gradient of `f`

w.r.t. `x`

.

For the sake of argument, let’s say that the size of `x`

is small (~10) and `w`

is large (1000 or more). This is not true for the above `f`

, but we can pretend.

One way I could approach this would be to define `g`

using ForwardDiff (since `x`

is small):

```
g = (x, w) -> ForwardDiff.gradient(x -> f(x, w), x)
```

and then take the jacobian of `g`

w.r.t. `w`

using ReverseDiff (since `w`

is large):

```
ReverseDiff.jacobian(w -> g(x, w), w)
```

but this fails with:

```
MethodError: Cannot `convert` an object of type ForwardDiff.Dual{2,Float64} to an object of type Float64
This may have arisen from a call to the constructor Float64(...),
since type constructors fall back to convert methods.
in increment_deriv! at /Users/rdeits/.julia/v0.5/ReverseDiff/src/derivatives/propagation.jl:34 [inlined]
in broadcast_increment_deriv!(::ReverseDiff.TrackedArray{Float64,Float64,1,Array{Float64,1},Array{Float64,1}}, ::Array{Float64,1}, ::Array{ForwardDiff.Dual{2,Float64},1}, ::CartesianIndex{1}, ::CartesianIndex{1}) at /Users/rdeits/.julia/v0.5/ReverseDiff/src/derivatives/propagation.jl:142
in special_reverse_exec!(::ReverseDiff.SpecialInstruction{Base.#.*,Tuple{Array{ForwardDiff.Dual{2,Float64},1},ReverseDiff.TrackedArray{Float64,Float64,1,Array{Float64,1},Array{Float64,1}}},ReverseDiff.TrackedArray{ForwardDiff.Dual{2,Float64},Float64,1,Array{ForwardDiff.Dual{2,Float64},1},Array{Float64,1}},Tuple{CartesianIndex{1},CartesianIndex{1}}}) at /Users/rdeits/.julia/v0.5/ReverseDiff/src/derivatives/elementwise.jl:473
in reverse_exec!(::ReverseDiff.SpecialInstruction{Base.#.*,Tuple{Array{ForwardDiff.Dual{2,Float64},1},ReverseDiff.TrackedArray{Float64,Float64,1,Array{Float64,1},Array{Float64,1}}},ReverseDiff.TrackedArray{ForwardDiff.Dual{2,Float64},Float64,1,Array{ForwardDiff.Dual{2,Float64},1},Array{Float64,1}},Tuple{CartesianIndex{1},CartesianIndex{1}}}) at /Users/rdeits/.julia/v0.5/ReverseDiff/src/tape.jl:93
in reverse_pass!(::Array{ReverseDiff.AbstractInstruction,1}) at /Users/rdeits/.julia/v0.5/ReverseDiff/src/tape.jl:87
in seeded_reverse_pass!(::Array{Float64,2}, ::Array{ReverseDiff.TrackedReal{ForwardDiff.Dual{2,Float64},Float64,Void},1}, ::ReverseDiff.TrackedArray{Float64,Float64,1,Array{Float64,1},Array{Float64,1}}, ::ReverseDiff.JacobianTape{##137#138,ReverseDiff.TrackedArray{Float64,Float64,1,Array{Float64,1},Array{Float64,1}},Array{ReverseDiff.TrackedReal{ForwardDiff.Dual{2,Float64},Float64,Void},1}}) at /Users/rdeits/.julia/v0.5/ReverseDiff/src/api/utils.jl:51
in seeded_reverse_pass! at /Users/rdeits/.julia/v0.5/ReverseDiff/src/api/tape.jl:47 [inlined]
in jacobian!(::Array{Float64,2}, ::ReverseDiff.JacobianTape{##137#138,ReverseDiff.TrackedArray{Float64,Float64,1,Array{Float64,1},Array{Float64,1}},Array{ReverseDiff.TrackedReal{ForwardDiff.Dual{2,Float64},Float64,Void},1}}, ::Array{Float64,1}) at /Users/rdeits/.julia/v0.5/ReverseDiff/src/api/jacobians.jl:122
in jacobian! at /Users/rdeits/.julia/v0.5/ReverseDiff/src/api/jacobians.jl:106 [inlined]
in jacobian(::Function, ::Array{Float64,1}, ::ReverseDiff.JacobianConfig{ReverseDiff.TrackedArray{Float64,Float64,1,Array{Float64,1},Array{Float64,1}},Void}) at /Users/rdeits/.julia/v0.5/ReverseDiff/src/api/jacobians.jl:25
in jacobian(::Function, ::Array{Float64,1}) at /Users/rdeits/.julia/v0.5/ReverseDiff/src/api/jacobians.jl:23
```

So, I have two questions:

- Am I just doing it wrong? Have I just forgotten everything I know about calculus?
- Should I expect running forwarddiff inside reversediff to work? Is there another way to use the tools to get what I want?

Oops, that was more than two questions.

Thanks!