# How to derivative using ForwardDiff

**URL:** <https://discourse.julialang.org/t/how-to-derivative-using-forwarddiff/96269>\
**Category:** General Usage\
**Created:** [March 18, 2023, 7:01am UTC](https://discourse.julialang.org/t/how-to-derivative-using-forwarddiff/96269 "2023-03-18T07:01:24Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![KotaAihara](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kotaaihara/32/43089_2.png) [@KotaAihara](https://discourse.julialang.org/u/KotaAihara)\
**Post date:** [March 18, 2023, 7:01am UTC](https://discourse.julialang.org/t/how-to-derivative-using-forwarddiff/96269/1 "2023-03-18T07:01:24Z")

</div>

I want any help. I want derivative a quadratic form function

```julia
using ForwardDiff

X,T = 80,52

param = [σ,μ]
function q(param::AbstractVector{T}) where T
    σ = param[1]
    V = zeros(T,T)
    for i in 1:T
        for j in 1:T
            V[i,j] = minimum([i,j]) * σ^2
        end
    end

    #buf_μ = Zygote.Buffer(μ)
    μ = param[2]
    μ_vec = zeros(T)
    for t in 1:T
        μ_vec[t] = t * μ
    end

    return μ_vec' * (V \ μ_vec)
end

q([1.0,1.0])

ForwardDiff.gradient(q,[1.0,1.0])

#isa(q_μ(μ), Union{Real,AbstractArray})

```

but it does not work. here is error message

> MethodError: no method matching Float64(::ForwardDiff.Dual{ForwardDiff.Tag{typeof(q), Float64}, Float64, 2})  
> Closest candidates are:  
> (::Type{T})(::Real, ::RoundingMode) where T\<:AbstractFloat at rounding.jl:200  
> (::Type{T})(::T) where T\<:Number at boot.jl:772  
> (::Type{T})(::AbstractChar) where T\<:Union{AbstractChar, Number} at char.jl:50  
> …  
> Stacktrace:  
> [1] convert(#unused#::Type{Float64}, x::ForwardDiff.Dual{ForwardDiff.Tag{typeof(q), Float64}, Float64, 2})  
> @ Base ./number.jl:7  
> [2] setindex!(::Matrix{Float64}, ::ForwardDiff.Dual{ForwardDiff.Tag{typeof(q), Float64}, Float64, 2}, ::Int64, ::Int64)  
> @ Base ./array.jl:968  
> [3] q(param::Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(q), Float64}, Float64, 2}})  
> @ Main ./In[3]:12  
> [4] vector\_mode\_dual\_eval!  
> @ ~/.julia/packages/ForwardDiff/vXysl/src/apiutils.jl:24 [inlined]  
> [5] vector\_mode\_gradient(f::typeof(q), x::Vector{Float64}, cfg::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(q), Float64}, Float64, 2, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(q), Float64}, Float64, 2}}})  
> @ ForwardDiff ~/.julia/packages/ForwardDiff/vXysl/src/gradient.jl:89  
> [6] gradient(f::Function, x::Vector{Float64}, cfg::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(q), Float64}, Float64, 2, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(q), Float64}, Float64, 2}}}, ::Val{true})  
> @ ForwardDiff ~/.julia/packages/ForwardDiff/vXysl/src/gradient.jl:0  
> [7] gradient(f::Function, x::Vector{Float64}, cfg::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(q), Float64}, Float64, 2, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(q), Float64}, Float64, 2}}}) (repeats 2 times)  
> @ ForwardDiff ~/.julia/packages/ForwardDiff/vXysl/src/gradient.jl:17  
> [8] top-level scope  
> @ In[3]:28

Could anyone tell me how to solve this problem?

---

<div class="post-metadata">

**Author:** ![simsurace](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simsurace/32/30216_2.png) [@simsurace](https://discourse.julialang.org/u/simsurace)\
**Post date:** [March 18, 2023, 7:08am UTC](https://discourse.julialang.org/t/how-to-derivative-using-forwarddiff/96269/2 "2023-03-18T07:08:48Z")

</div>

You have a confusion between the type parameter and variable name `T`.  
I think you want `T` to be the length of the vector. But `T` in `AbstractVector{T}` is the element type.

---

<div class="post-metadata">

**Author:** ![braamvandyk](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/braamvandyk/32/5086_2.png) [@braamvandyk](https://discourse.julialang.org/u/braamvandyk)\
**Post date:** [March 18, 2023, 7:26am UTC](https://discourse.julialang.org/t/how-to-derivative-using-forwarddiff/96269/3 "2023-03-18T07:26:14Z")

</div>

Quite a few things here…

You define an array to hold σ and μ, but they have not been defined at this point, so this line errors. I’m going to guess you meant that line to be a comment, which should then start with a #.

As mentioned, you are confusing the type parameter, T, with a size stored in a variable T. You don’t really need parameterized types and using an AbstractVector is probably not a great idea either.

With those fixed, your function at least executes:

```julia
using ForwardDiff

X,T = 80,52

# param = [σ,μ]
function q(param)
    σ = param[1]
    V = zeros(T,T)
    for i in 1:T
        for j in 1:T
            V[i,j] = minimum([i,j]) * σ^2
        end
    end

    #buf_μ = Zygote.Buffer(μ)
    μ = param[2]
    μ_vec = zeros(T)
    for t in 1:T
        μ_vec[t] = t * μ
    end

    return μ_vec' * (V \ μ_vec)
end

q([1.0,1.0])

```

I’ll leave comments on ForwardDiff’s ability to handle your function to more knowledgeable people.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [March 18, 2023, 8:23am UTC](https://discourse.julialang.org/t/how-to-derivative-using-forwarddiff/96269/4 "2023-03-18T08:23:17Z")

</div>

And here is a version that actually works:

```julia
using ForwardDiff

function q(param; T)
    σ, μ = param
    R = eltype(param)
    V = zeros(R, T, T)
    μ_vec = zeros(R, T)
    for i in 1:T, j in 1:T
        V[i,j] = min(i, j) * σ^2
    end
    for i in 1:T
        μ_vec[i] = i * μ
    end
    return μ_vec' * (V \ μ_vec)
end

q([1.0, 1.0]; T=52)
ForwardDiff.gradient(param -> q(param; T=52), [1.0, 1.0])

```

The reason why ForwardDiff.jl errored is because it needs your function to work with arbitrary subtypes of `Real`, which includes so-called “dual numbers”. But when you create `V` and `μ_vec`, you implicitly declare their elements to be of type `Float64` (because `zeros(k)` is equivalent to `zeros(Float64, k)`). As a result, when `param` has elements of type `Dual`, Julia doesn’t know how to convert them back to `Float64` to store things like `min(i, j) * σ^2` in your array.  
How did I fix this? By specifying that the elements of `V` and `μ_vec` should have the same type as the elements of `param`, which I denoted by `R`. This is an instance of generic programming, and you’ll run into it quite often in Julia.

Other minor remarks:

- You don’t need to allocate an array `[i, j]` when computing the minimum of `i` and `j`: the function `min(i, j)` works for two arguments and will be much faster
- You don’t seem to use `X` so I removed it
- I gave `T` as a keyword argument to the function, whereas in your code it was a global variable (which can negatively impact performance)

---

<div class="post-metadata">

**Author:** ![KotaAihara](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kotaaihara/32/43089_2.png) [@KotaAihara](https://discourse.julialang.org/u/KotaAihara)\
**Post date:** [March 27, 2023, 8:49am UTC](https://discourse.julialang.org/t/how-to-derivative-using-forwarddiff/96269/5 "2023-03-27T08:49:32Z")

</div>

Thanks a lot for your help.  
It worked!!

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [March 27, 2023, 10:18am UTC](https://discourse.julialang.org/t/how-to-derivative-using-forwarddiff/96269/6 "2023-03-27T10:18:23Z")

</div>

Great! Could you mark my answer as solution, so that others know they don’t have to weigh in?