Optim.jl wrongly reporting success

Albert_Zevelev
Mon, 11 Jan 2021 18:59:30 +0000
Optimization (Mathematical)

I’m solving for the MLE.

using Distributions, Random, Optim;
DGP_True = LogNormal(17,7);
Random.seed!(123);
const d_train = rand(DGP_True, 1_000)
const d_test  = rand(DGP_True, 1_000)

# LogNormal
function loglike(θ)
    dist          = LogNormal(θ[1],θ[2])
    terms         = logpdf.(dist,d_train)
    loglikelihood = sum(terms)
    return -loglikelihood #Min(-LogLik) =Max(LogLik)
end
#
optimum = optimize(loglike,ones(2))
Optim.minimizer(optimum)
Distributions.fit_mle(LogNormal,d_train)

# Optim.jl &  Distributions.jl solutions are very close.


# Normal
function loglike(θ)
    dist          = Normal(θ[1],θ[2])
    terms         = logpdf.(dist,d_train)
    loglikelihood = sum(terms)
    return -loglikelihood #Min(-LogLik) =Max(LogLik)
end
#
optimum = optimize(loglike,ones(2))
Optim.minimizer(optimum)
Distributions.fit_mle(Normal,d_train)

# Optim.jl reports "sucess" BUT
# Optim.jl &  Distributions.jl solutions are very different!

A second example:

using Distributions, Random, Optim;
DGP_True = LogNormal(0,1);
Random.seed!(123);
const d_train = rand(DGP_True, 1_000);
const d_test  = rand(DGP_True, 1_000);

D1=Distributions.fit_mle(Laplace, d_train)

function loglike(θ)
    dist          = Laplace(θ[1],θ[2])
    terms         = logpdf.(dist,d_train)
    loglikelihood = sum(terms)
    return -loglikelihood #Min(-LogLik) =Max(LogLik)
end
sol=optimize(loglike,ones(2),Optim.Options(iterations=10^4))
D2=Optim.minimizer(sol)

The estimate for the first parameter is very similar:
μ=1.0112216426398657 vs Optim’s 1.0116066436206388
The estimate for the 2nd parameter is different:
θ=0.9005083163386765 vs Optim’s 1.1872092883101257

My main concern is not that Optim.jl is not able to solve this, it is that it is wrongly reporting “success”.

I created an issue, but it was closed for some reason…

5 posts - 5 participants