# Gradient using ForwardDiff.jl not working

**URL:** https://discourse.julialang.org/t/gradient-using-forwarddiff-jl-not-working/123193
**Category:** General Usage
**Tags:** forwarddiff, autodiff, gradient
**Created:** [November 28, 2024, 1:38am UTC](https://discourse.julialang.org/t/gradient-using-forwarddiff-jl-not-working/123193 "2024-11-28T01:38:31Z")
**Posts on this page:** 1
**Showing post:** 2

<div class="post-metadata">

### Author: ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)
#### Post date: [November 28, 2024, 5:53am UTC](https://discourse.julialang.org/t/gradient-using-forwarddiff-jl-not-working/123193/2 "2024-11-28T05:53:32Z")

</div>

Hi @miguelborrero!  
The reason for this error is explained in the [limitations of ForwardDiff](https://juliadiff.org/ForwardDiff.jl/stable/user/limitations/): for it to work, you must make sure that your functions can accept numbers of generic type, not just `Float64`. Your code in its current state has several forced conversions to `Float64`: they do not improve performance and they make autodiff in forward mode impossible.  
Here is a version where I made a few changes to:

- increase generality by adapting to arbitrary types
- improve clarify and possibly performance by turning `Observation` into a `StaticVector` instead of having a loop over its field names (this only helps when `Observation` has a handful of fields)

```julia
using ForwardDiff
using LinearAlgebra
using StaticArrays

# adapt struct to arbitrary field type
struct Observation{T<:Real}
    y::T
    x1::T
    x2::T
end

# efficiently turn Observation into a vector of statically known size
to_vector(o::Observation) = SVector(o.y, o.x1, o.x2)

# this function will generate the logit data for Q4 
function simulate_logit_data(; n_obs=300000)
    # define the DGP 
    X1 = 1.0:0.01:10.0
    X2 = 18:0.01:65.0
    α = -1.0
    θ_1 = 0.5
    θ_2 = 0.02
    # pre-allocate the container for the data, fully specifying its type
    data = Vector{Observation{Float64}}(undef, n_obs)
    for i in eachindex(data)
        x1 = rand(X1)
        x2 = rand(X2)
        u = rand()
        y = (α + θ_1 * x2 + θ_2 * x2 + log(u / (1 - u)) > 0) ? 1.0 : 0.0
        data[i] = Observation(y, x1, x2)
    end
    return data
end

# this function will compute the log-likelihood given a parameter space point 
function LL(θ, data)
    ll = zero(eltype(θ)) # give ll the element type of theta, which may be a Dual number
    for i in eachindex(data)
        obs = data[i]
        Xθ = dot(θ, to_vector(obs)) # precompute this quantity as a dot product
        ll += -log(1 + exp(Xθ)) + obs.y * Xθ
    end
    return ll
end

data = simulate_logit_data()
ForwardDiff.gradient(Base.Fix2(LL, data), @SVector(rand(3)))

```

Related:

> [@Understanding the output of @code\_warntype](https://discourse.julialang.org/t/understanding-the-output-of-code-warntype/123191):
>
> Hi there, I simulated some data which I stored as an Array{Observation} where Observation is a composite type that stores the elements of each observation. I then I wrote a function to compute the log-likelihood of the data as: struct Observation{T} y::T x1::T x2::T end # this function will generate the logit data for Q4 function simulate\_logit\_data(; n\_obs = 300000) # define the DGP X1 = 1.0:0.01:10.0 X2 = 18:0.01:65.0 α = -1.0 θ\_1 = 0.5 θ\_2 = 0.02 …

---

_[View the full topic](https://discourse.julialang.org/t/gradient-using-forwarddiff-jl-not-working/123193)._