# Can't figure out why log-likelihood function doesn't work with autodiff?

**URL:** <https://discourse.julialang.org/t/cant-figure-out-why-log-likelihood-function-doesnt-work-with-autodiff/97636>\
**Category:** New to Julia\
**Tags:** question\
**Created:** [April 19, 2023, 12:13am UTC](https://discourse.julialang.org/t/cant-figure-out-why-log-likelihood-function-doesnt-work-with-autodiff/97636 "2023-04-19T00:13:45Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![maxdrohde](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maxdrohde/32/47856_2.png) [@maxdrohde](https://discourse.julialang.org/u/maxdrohde)\
**Post date:** [April 19, 2023, 12:13am UTC](https://discourse.julialang.org/t/cant-figure-out-why-log-likelihood-function-doesnt-work-with-autodiff/97636/1 "2023-04-19T00:13:45Z")

</div>

I’m trying to practice Julia by coding an ordinal regression model. I have defined a log-likelihood function and it works as expected when I used `Optim.jl` for optimization. To gain additional speed, I want to try using the autodiff features in Julia. However, they don’t seem to work with my log-likelihood function and I can’t figure out why. Any help is appreciated.

## Log-Likelihood code

```julia-auto
function neg_loglik(params::Vector, X::Matrix, Y_ord::Vector)

    # Define the link function
    F(x) = cdf(Normal(0,1), x)

    # Number of categories
    K = maximum(Y_ord)

    # Define the subset of the params vector for the cutpoints and the betas
    c = params[1:(K-1)]
    β = params[K:end]

    # Number of observations
    n = length(Y_ord)

    # Initialize a vector to store likelihood values
    L = zeros(n)

    # Return Inf negative log-likelihood if the cutpoints are out of order
    if !issorted(c)
        return(Inf)
    end

    for i in 1:n
        # Current observation
        y = Y_ord[i]
        # Linear predictor
        η = dot(X[i,:], β)

        if y == 1
            L[i] = F(c[1] - η)
        elseif y == K
            L[i] = 1 - F(c[K - 1] - η)
        else
            L[i] = F(c[y] - η) - F(c[y-1] - η)
        end
    end

    ℓ = log.(L)

    return(-sum(ℓ))
end

neg_loglik_c(params::Vector) = neg_loglik(params, X, Y_ord)

```

## Trying autodiff

```julia-auto
ForwardDiff.gradient(neg_loglik_c, vcat(sort!(rand(K)), zeros(p)))

```

## Error message

```julia-auto
ERROR: MethodError: no method matching Float64(::ForwardDiff.Dual{ForwardDiff.Tag{typeof(neg_loglik_c), Float64}, Float64, 7})
Closest candidates are:
  (::Type{T})(::Real, ::RoundingMode) where T<:AbstractFloat at rounding.jl:200
  (::Type{T})(::T) where T<:Number at boot.jl:772
  (::Type{T})(::AbstractChar) where T<:Union{AbstractChar, Number} at char.jl:50
  ...
Stacktrace:
 [1] convert(#unused#::Type{Float64}, x::ForwardDiff.Dual{ForwardDiff.Tag{typeof(neg_loglik_c), Float64}, Float64, 7})
   @ Base ./number.jl:7
 [2] setindex!(A::Vector{Float64}, x::ForwardDiff.Dual{ForwardDiff.Tag{typeof(neg_loglik_c), Float64}, Float64, 7}, i1::Int64)
   @ Base ./array.jl:966
 [3] neg_loglik(params::Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(neg_loglik_c), Float64}, Float64, 7}}, X::Matrix{Float64}, Y_ord::Vector{Int64})
   @ Main ./code:36
 [4] neg_loglik_c(params::Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(neg_loglik_c), Float64}, Float64, 7}})
   @ Main ./code:88
 [5] chunk_mode_gradient(f::typeof(neg_loglik_c), x::Vector{Float64}, cfg::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(neg_loglik_c), Float64}, Float64, 7, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(neg_loglik_c), Float64}, Float64, 7}}})
   @ ForwardDiff ~/.julia/packages/ForwardDiff/vXysl/src/gradient.jl:123
 [6] gradient(f::Function, x::Vector{Float64}, cfg::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(neg_loglik_c), Float64}, Float64, 7, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(neg_loglik_c), Float64}, Float64, 7}}}, ::Val{true})
   @ ForwardDiff ~/.julia/packages/ForwardDiff/vXysl/src/gradient.jl:21
 [7] gradient(f::Function, x::Vector{Float64}, cfg::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(neg_loglik_c), Float64}, Float64, 7, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(neg_loglik_c), Float64}, Float64, 7}}}) (repeats 2 times)
   @ ForwardDiff ~/.julia/packages/ForwardDiff/vXysl/src/gradient.jl:17
 [8] top-level scope
   @ REPL[7]:1

```

---

<div class="post-metadata">

**Author:** ![johnmyleswhite](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnmyleswhite/32/31_2.png) [@johnmyleswhite](https://discourse.julialang.org/u/johnmyleswhite)\
**Post date:** [April 19, 2023, 12:37am UTC](https://discourse.julialang.org/t/cant-figure-out-why-log-likelihood-function-doesnt-work-with-autodiff/97636/2 "2023-04-19T00:37:14Z")

</div>

Usually that error comes up because you’re trying to convert something to Float64 somewhere in your code that needs to accept Dual numbers to work with ForwardDiff. Not obvious to me in your code where that’s happening, but I would look for that.

---

<div class="post-metadata">

**Author:** ![cgeoga](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cgeoga/32/216186_2.png) [@cgeoga](https://discourse.julialang.org/u/cgeoga)\
**Post date:** [April 19, 2023, 1:05am UTC](https://discourse.julialang.org/t/cant-figure-out-why-log-likelihood-function-doesnt-work-with-autodiff/97636/3 "2023-04-19T01:05:32Z")

</div>

It’s the `zeros(n)`. Try instead `zeros(eltype(params), n)` and see how it goes.