Loglikelihood vs logpdf

Specific Domains Statistics
1

What is the relationship between StatsAPI.loglikelihood(d, x) and Distributions.logpdf(d, x) where d::Distribution and x is a draw from it?

In the examples I’ve tried, they give the same results.

let d = Normal(0,1)
    x = .3
    logpdf(d, x) == loglikelihood(d,x) # true
end

Are they the same? When should I use each? For instance, should I call loglikelihood(d,x) inside of my own composite log-likelihood functions, and use logpdf(d,x) inside of prior-probability functions?

1 Like
2

The main difference is that loglikelihood computes the sum of the log likelihoods and logpdf computes the log likelihood of a single observation. Your example is a special case in which the outputs are the same.

using Distributions 

dist = Normal(0, 1)
x = rand(dist, 10)

logpdf.(dist, x)

output

10-element Vector{Float64}:
 -1.1116329495246389
 -0.9190086195213015
 -0.9432279155524945
 -1.0348876720528215
 -1.0466074563159673
 -1.7416591504427266
 -1.294146605795191
 -1.2890879707921559
 -2.0101113445525365
 -0.9189482910005295

loglikelihood(dist, x)
output
-12.309317975550362

The distinction would be clearer if the name sumloglikelihood was used instead.

2 Likes
3

If you call them the same way, broadcasting both, they give the same results.

julia> logpdf.(dist, x)
10-element Vector{Float64}:
 -1.0203926929373632
 -0.9234209417660356
 -0.9344046575034405
 -1.0199860243797683
 -3.1309569251593743
 -1.0147293928603724
 -1.1854557641508374
 -1.5117326368202066
 -1.2205551629592766
 -1.2778617723196464


julia> loglikelihood.(dist, x)
10-element Vector{Float64}:
 -1.0203926929373632
 -0.9234209417660356
 -0.9344046575034405
 -1.0199860243797683
 -3.1309569251593743
 -1.0147293928603724
 -1.1854557641508374
 -1.5117326368202066
 -1.2205551629592766
 -1.2778617723196464

In statistics, when estimating parameters, the term likelihood is used to speak of the distribution of the data, but evaluated at some point in the parameter space, which is the set of parameter values under consideration. The actual distribution is simply the likelihood, but evaluated at the point in the parameter space that corresponds to the true, often unknown, parameter vector. So, the distinction is one of the name only. When doing statistics, it is called the likelihood. When doing probability theory, it is called the pdf.

2 Likes