# Quasi Maximum Likelihood implementation with Optim.jl

**URL:** <https://discourse.julialang.org/t/quasi-maximum-likelihood-implementation-with-optim-jl/9757>\
**Category:** Optimization (Mathematical)\
**Created:** [March 16, 2018, 12:53pm UTC](https://discourse.julialang.org/t/quasi-maximum-likelihood-implementation-with-optim-jl/9757 "2018-03-16T12:53:35Z")\
**Posts on this page:** 2\
**Page:** 2

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [April 3, 2018, 6:06am UTC](https://discourse.julialang.org/t/quasi-maximum-likelihood-implementation-with-optim-jl/9757/21 "2018-04-03T06:06:45Z")

</div>

> [@AnttiSim](#):
>
> The problem is that with a system of censored equation the solution is not invariant to which equation you omit.

I am not sure I understand this. I did not suggest omitting an equation, just a way to map a subset of \mathbb{R}^n to the feasible parameter space (with the constraint).

> [@AnttiSim](#):
>
> I’ve found a solution in literature that uses Gibbs sampling and I’m now trying to get my head around it.

I would suspect that modern HMC methods would do much better than Gibbs (unless it is a very specialized problem). See [DynamicHMC.jl](https://github.com/tpapp/DynamicHMC.jl), or the NUTS algorithms in [Klara.jl](https://github.com/JuliaStats/Klara.jl) and [Mamba.jl](https://github.com/brian-j-smith/Mamba.jl).

---

<div class="post-metadata">

**Author:** ![anriseth](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/anriseth/32/3106_2.png) [@anriseth](https://discourse.julialang.org/u/anriseth)\
**Post date:** [April 6, 2018, 8:45am UTC](https://discourse.julialang.org/t/quasi-maximum-likelihood-implementation-with-optim-jl/9757/22 "2018-04-06T08:45:14Z")

</div>

> [@AnttiSim](#):
>
> At this point I realized that I need to include constraints to parameters and that isn’t possible with Optim.jl

We have a constrained Newton solver at [ConstrainedOptim](https://github.com/JuliaNLSolvers/ConstrainedOptim.jl). This could work if evaluating your gradient, Hessian (and derivatives of the constraint) is not too demanding.

Otherwise, try to add a penalty to your function such as  
`f(x) = g(x) + lambda * sum(abs2, delta)`, and then use a [continuation method](https://en.wikipedia.org/wiki/Numerical_continuation). So you optimize `f` for a small value of `lambda`, then restart the optimization with a larger value of `lambda`, starting from the minimizer you got from the first run, and then repeat.

[Previous page](https://discourse.julialang.org/t/quasi-maximum-likelihood-implementation-with-optim-jl/9757.md?page=1)