Get MLE parameters (e.g., p-value, confidence intervals...) from a Turing model using Optim.jl

Specific Domains Probabilistic programming
, , ,
3

Also worth mentioning this post that showcases the use of coeftable() on the output of optimize.

For those interested, here is an attempt at getting a more complete version (with p-values and CIs via a z-approximation) (also because I have issues with coeftable doesn’t work)

function coeftable2(model)
    mle = Optim.optimize(model, MLE())
    params = DataFrame(
        Name=coefnames(mle),
        Coef=mle.values,
        StdError=stderror(mle)
    )
    params[!, :z] = params[!, :Coef] ./ params[!, :StdError]
    params[!, :p] = 2 * (1 .- cdf.(Normal(0, 1), abs.(params[!, :z])))
    params[!, :CI_low] = params[!, :Coef] .+ quantile(Normal(0, 1), 0.025) .* params[!, :StdError]
    params[!, :CI_high] = params[!, :Coef] .+ quantile(Normal(0, 1), 0.975) .* params[!, :StdError]
    return params
end

Which works like this:

using Turing
using DataFrames
using LinearAlgebra
using StatsBase
using Distributions

function generate_data(a, b, v, nr_samples)
    x = float.(collect(1:nr_samples))
    y = a .* x .+ b .+ randn(nr_samples) .* sqrt(v)
    return x, y
end

x, y = generate_data(0.3, 25.0, 100.0, 40)

@model function model_lm(y, x)
    # Set variance prior.
    σ² ~ truncated(Normal(0, 100); lower=0)
    intercept ~ Normal(0, sqrt(3))
    coefficient ~ TDist(3)

    # Calculate all the mu terms.
    mu = intercept .+ x * coefficient
    y ~ MvNormal(mu, σ² * LinearAlgebra.I)
end

# MLE
model = model_lm(y, x)
coeftable2(model)

3×7 DataFrame
 Row │ Name         Coef        StdError   z        p            CI_low    CI_high    
     │ Symbol       Float64     Float64    Float64  Float64      Float64   Float64
─────┼────────────────────────────────────────────────────────────────────────────────
   1 │ σ²           118.147     26.4185    4.47214  7.74422e-6   66.3679   169.927
   2 │ intercept     22.2225     3.50273   6.34435  2.23373e-10  15.3573    29.0878
   3 │ coefficient    0.482927   0.148884  3.24364  0.00118011    0.19112    0.774734