# Julia vs R vs Python: simple MLE problem
**URL:** https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012
**Category:** Optimization (Mathematical)
**Created:** [February 12, 2018, 1:16pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012 "2018-02-12T13:16:37Z")
**Posts on this page:** 20
**Page:** 2
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [February 13, 2018, 7:58pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/21 "2018-02-13T19:58:33Z")
> [@xiaodai](#):
>
> I have been thinking about whether I would prefer to use Julia or R in this instance. For work, I have written code to solve a much more complicated version of the problem in R. That’s before I started using Julia seriously.
Is there an example of someone defining their own number type in R? I don’t know of an example other than RMPFR which is quite complicated, but don’t know of things like bindings for Arb in R, or how it can be used in the standard library functions.
---
### Author:  [@Elrod](https://discourse.julialang.org/u/Elrod)
#### Post date: [February 13, 2018, 8:07pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/22 "2018-02-13T20:07:31Z")
I looked at Rmpfr briefly, and didn’t see a way to invert matrices.
They also need to implement their own uniroot, integrate, etc…
I know last summer a professor in my university was interested in tropical geometry, where `+` is `min` and `*` is `+`. I don’t know what the applications are. I think the goal was to sample from a space of phylogenetic trees. Getting creative like that would be a lot easier in Julia.
---
### Author:  [@pkofod](https://discourse.julialang.org/u/pkofod)
#### Post date: [February 13, 2018, 8:43pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/23 "2018-02-13T20:43:44Z")
By the way, did you use (L)BFGS in Python? There seems to be an inverse Hessian approximation in the output as well as a gradient.
---
### Author:  [@xiaodai](https://discourse.julialang.org/u/xiaodai)
#### Post date: [February 13, 2018, 10:45pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/24 "2018-02-13T22:45:33Z")
I honestly don’t know what that means and/or why it might be important. I just know I can specify bounds 😅
---
### Author:  [@pkofod](https://discourse.julialang.org/u/pkofod)
#### Post date: [February 14, 2018, 6:16am UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/25 "2018-02-14T06:16:29Z")
Ah, okay. It’s the optimization algorithm.
---
### Author:  [@tyleransom](https://discourse.julialang.org/u/tyleransom)
#### Post date: [February 14, 2018, 3:14pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/26 "2018-02-14T15:14:35Z")
This is really cool; I’ve been looking for a good way to make an fminunc/con mirror in Julia so this is great to see! I may be in touch with a PR on your GitHub repository at some point in the future.
---
### Author:  [@mcreel](https://discourse.julialang.org/u/mcreel)
#### Post date: [February 14, 2018, 4:53pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/27 "2018-02-14T16:53:23Z")
Thanks, PRs are welcome. Note that the fminunc/con bits are just frontends to Optim and NLOpt, which do the real work. They are intended to ease the transition for students who are used to Matlab.
---
### Author:  [@tyleransom](https://discourse.julialang.org/u/tyleransom)
#### Post date: [February 14, 2018, 9:09pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/28 "2018-02-14T21:09:49Z")
Sounds good. I’ve been in search of a good interface for Julia to do common econometric MLE problems (stuff that’s not in GLM, e.g. multinomial logit, etc.). I’ve tried implementing in JuMP but the Hessian computation was overly burdensome (in terms of memory usage—see [this discourse](https://discourse.julialang.org/t/jump-excessive-memory-usage/1057/15)). So it seems like using Optim instead of JuMP may be an improvement.
---
### Author:  [@mcreel](https://discourse.julialang.org/u/mcreel)
#### Post date: [February 15, 2018, 6:57am UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/29 "2018-02-15T06:57:02Z")
This should be pretty easy, in principle. You just need to set the likelihood function following this example:
```julia
model = β -> poisson(β, y, x)
mleresults(model, β, "simple ML example");
```
where the likelihood returns the log likelihood for each observation, in an array. Example likelihood functions are in [https://github.com/mcreel/Econometrics.jl/tree/master/src/ML/Likelihoods](https://github.com/mcreel/Econometrics.jl/tree/master/src/ML/Likelihoods).
mleresults() actually uses LBFGS from Optim for the maximization. Currently, there’s no overt support for constrained ML, but that would be pretty easy to add.
---
### Author:  [@miles.lubin](https://discourse.julialang.org/u/miles.lubin)
#### Post date: [February 15, 2018, 2:36pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/30 "2018-02-15T14:36:32Z")
**Edited**
Using JuMP shouldn’t be conflated with using methods that require second-order derivatives. JuMP is perfectly happy to skip the Hessian computation if the solver doesn’t ask for it. In this case you won’t see the corresponding memory usage. You could use JuMP to compute first-order derivatives for the methods in Optim that need them, but JuMP isn’t really targeting the unconstrained case so this link hasn’t been well developed, and as far as I know the AD solutions in Optim are good enough already.
---
### Author:  [@tyleransom](https://discourse.julialang.org/u/tyleransom)
#### Post date: [February 15, 2018, 4:13pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/31 "2018-02-15T16:13:01Z")
Thanks, Miles! I was unaware that Optim didn’t have an AD option; that’s really good to know and makes me want to go back to using JuMP.
I was under the impression that `NLobjective` (which is the workhorse for MLE models) requires the Hessian for optimization. Is that correct?
And, if not, is it possible to compute the Hessian at the optimum post-estimation? I really only need it at the end for doing statistical inference.
---
### Author:  [@pkofod](https://discourse.julialang.org/u/pkofod)
#### Post date: [February 15, 2018, 4:22pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/32 "2018-02-15T16:22:41Z")
> [@tyleransom](#):
>
> I was unaware that Optim didn’t have an AD option; that’s really good to know and makes me want to go back to using JuMP.
This is false, and I’m not sure that’s what Miles is implying either.
---
### Author:  [@miles.lubin](https://discourse.julialang.org/u/miles.lubin)
#### Post date: [February 15, 2018, 4:36pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/33 "2018-02-15T16:36:23Z")
My post was hastily written, I’ve edited it.
---
### Author:  [@mcreel](https://discourse.julialang.org/u/mcreel)
#### Post date: [February 15, 2018, 5:57pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/34 "2018-02-15T17:57:08Z")
An example of computing the Hessian post optimization, using the Calculus.jl package, is here: [https://github.com/mcreel/Econometrics.jl/blob/master/src/ML/mle.jl](https://github.com/mcreel/Econometrics.jl/blob/master/src/ML/mle.jl), line 21.
---
### Author:  [@tyleransom](https://discourse.julialang.org/u/tyleransom)
#### Post date: [February 15, 2018, 6:13pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/35 "2018-02-15T18:13:09Z")
Thanks for the clarification, @miles.lubin and @pkofod!
---
### Author:  [@pkofod](https://discourse.julialang.org/u/pkofod)
#### Post date: [February 15, 2018, 6:22pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/36 "2018-02-15T18:22:48Z")
You have Optim loaded already, so you could also create an
```julia
td = TwiceDifferentiable(obj, θ; autodiff = :forward)
value_gradient!(td, θ)
hessian!(td, θ)
L, dL, d2L = value(td), gradient(td), hessian(td)
```
though if this is horrible depends on the user I guess 🙂 It’s sort of just based around the way Optim does this stuff internally.
---
### Author:  [@xiaodai](https://discourse.julialang.org/u/xiaodai)
#### Post date: [April 5, 2019, 12:07pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/37 "2019-04-05T12:07:38Z")
Happy to report that the compilation latency with Optim in Julia 1.1 has gone from 7.5 seconds to sub 1s for me now.
So I’ve updated the post
[https://www.codementor.io/zhuojiadai/julia-vs-r-vs-python-simple-optimization-gnqi4njro](https://www.codementor.io/zhuojiadai/julia-vs-r-vs-python-simple-optimization-gnqi4njro)
Now Julia is the clear winner for me when doing simple optimisation problems.
---
### Author:  [@ianfiske](https://discourse.julialang.org/u/ianfiske)
#### Post date: [April 5, 2019, 1:11pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/38 "2019-04-05T13:11:14Z")
Wow! Any idea what reduced the compilation latency of Optim? Some of my own private packages tend to suffer from long compilation times and I’ve wondered if there are standard strategies to improve compilation time.
---
### Author:  [@xiaodai](https://discourse.julialang.org/u/xiaodai)
#### Post date: [April 5, 2019, 1:23pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/39 "2019-04-05T13:23:55Z")
> [@pkofod](#):
>
> Compile times. I don’t really think we (Optim.jl) can do so much about that currently
Something’s happened to Optim compile times right?
---
### Author:  [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)
#### Post date: [April 5, 2019, 1:26pm UTC](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012/40 "2019-04-05T13:26:34Z")
A lot of work has gone into improving compilation and startup times: [Issues · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues?utf8=%E2%9C%93&q=+label%3Alatency+).
[Previous page](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012.md?page=1)
[Next page](https://discourse.julialang.org/t/julia-vs-r-vs-python-simple-mle-problem/9012.md?page=3)