# Optim.jl can't optimize function which includes BigFloat variables

**URL:** <https://discourse.julialang.org/t/optim-jl-cant-optimize-function-which-includes-bigfloat-variables/65095>\
**Category:** General Usage\
**Tags:** question, package, optim, optimization\
**Created:** [July 22, 2021, 9:16am UTC](https://discourse.julialang.org/t/optim-jl-cant-optimize-function-which-includes-bigfloat-variables/65095 "2021-07-22T09:16:37Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Strange\_Xue](https://avatars.discourse-cdn.com/v4/letter/s/e9a140/32.png) [@Strange\_Xue](https://discourse.julialang.org/u/Strange_Xue)\
**Post date:** [July 22, 2021, 9:16am UTC](https://discourse.julialang.org/t/optim-jl-cant-optimize-function-which-includes-bigfloat-variables/65095/1 "2021-07-22T09:16:37Z")

</div>

This is a simple maximum likelihood estimation example from the tutorials of the Optm.jl, I just change the the log\_ \sigma 's type from `float` to `BigFloat` in the `Log_Likelihood` function , however, the error message shows that `no method matching big`, I know there’s no error if it is `float` type, but sometime, there exists some questions including really big numbers, so I wonder, is there some way julia can optimize a function which includes `BigFloat`?

```julia
using Optim, NLSolversBase, Random
using LinearAlgebra: diag
Random.seed!(0); # Fix random seed generator for reproducibility

n = 500 # Number of observations
nvar = 2 # Number of variables
β = ones(nvar) * 3.0 # True coefficients
x = [ones(n) randn(n, nvar - 1)] # X matrix of
ε = randn(n) * 0.5 # Error variance
y = x * β + ε; # Generate Data
function Log_Likelihood(X, Y, β, log_σ)
    σ = exp(big(log_σ))
    llike = -n/2*log(2π) - n/2* log(σ^2) - (sum((Y - X * β).^2) / (2σ^2))
    llike = -llike
end
func = TwiceDifferentiable(vars -> Log_Likelihood(x, y, vars[1:nvar], vars[nvar + 1]),
                           ones(nvar+1); autodiff=:forward)
opt = optimize(func, ones(nvar+1))
parameters = Optim.minimizer(opt)
# ERROR: LoadError: MethodError: no method matching big(::ForwardDiff.Dual.....

```

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [July 22, 2021, 9:21am UTC](https://discourse.julialang.org/t/optim-jl-cant-optimize-function-which-includes-bigfloat-variables/65095/2 "2021-07-22T09:21:54Z")

</div>

> [@Strange\_Xue](#):
>
> `big(log_σ)`

try passing big numbers directly instead of `big()` it inside, the error seems to be `Forwarddiff` doesn’t specialize `big()` on their Dual number. But they have promotions rules and everything

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [July 22, 2021, 10:06am UTC](https://discourse.julialang.org/t/optim-jl-cant-optimize-function-which-includes-bigfloat-variables/65095/3 "2021-07-22T10:06:25Z")

</div>

As an aside, do you really need `BigFloat` or is something like [DoubleFloats.jl](https://juliahub.com/ui/Packages/DoubleFloats/KzTCm/1.1.23) sufficient for your purposes?

Other than that, passing `log_σ` as a `BigFloat` directly instead of creating it inside of your function is going to be better as @jling mentioned.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [July 22, 2021, 12:34pm UTC](https://discourse.julialang.org/t/optim-jl-cant-optimize-function-which-includes-bigfloat-variables/65095/4 "2021-07-22T12:34:00Z")

</div>

> [@Strange\_Xue](#):
>
> ```julia
> function Log_Likelihood(X, Y, β, log_σ)
> σ = exp(big(log_σ))
> llike = -n/2*log(2π) - n/2* log(σ^2) - (sum((Y - X * β).^2) / (2σ^2))
> llike = -llike
> end
> 
> ```

There is no point in converting a single term like `σ` to `BigFloat` if the rest of the computations are done in double precision. e.g. `log(2π)` is computed in double precision, and that will generally limit your accuracy to double precision even if _parts_ of the computation are `BigFloat`.

(Aside: [don’t use global variables](https://docs.julialang.org/en/v1/manual/performance-tips/#Avoid-global-variables) like `n` in your functions if you can help it. In this case, you should get `n = length(Y)` from the arguments.)

In general, you should try to write code that works for any number type _based on the types of the arguments_. This can be a little tricky but the result is much more flexible code. For example:

```julia
function Log_Likelihood2(X, Y, β, log_σ)
    n = length(Y)
    σ = exp(log_σ)
    llike = n * log_σ + sum((Y .- X * β).^2) / 2σ^2
    return llike + n * log(2*oftype(llike, π)) / 2
end

```

Not that it uses `oftype(llike, π)` to compute `π` in the same precision as `llike`, and rearranges the calculation to avoid `n/2` (`Int`/`Int` uses double precision, though technically dividing integers by `2` can be done exactly it still changes the precision).

Now you can compute in whatever precision you want — but you should probably pass _all_ of the arguments in the same precision, as otherwise the accuracy will be limited by the least-precise argument.

This is also important for automatic differentiation, since that works by passing “just another number type” (`ForwardDiff.Dual`) that [keeps track of derivatives](https://arxiv.org/abs/1607.07892).

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [July 22, 2021, 12:52pm UTC](https://discourse.julialang.org/t/optim-jl-cant-optimize-function-which-includes-bigfloat-variables/65095/5 "2021-07-22T12:52:08Z")

</div>

> [@Strange\_Xue](#):
>
> ```julia
> # no method matching big(::ForwardDiff.Dual.....
> 
> ```

On a separate note (since this is not the right solution to your problem as noted above), it seems like `big(::Dual)` should work. I filed an issue: [missing method big(::Dual) · Issue #537 · JuliaDiff/ForwardDiff.jl · GitHub](https://github.com/JuliaDiff/ForwardDiff.jl/issues/537)

This would make an easy [first PR](https://github.com/firstcontributions/first-contributions) if someone wants to take it on.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [July 22, 2021, 1:03pm UTC](https://discourse.julialang.org/t/optim-jl-cant-optimize-function-which-includes-bigfloat-variables/65095/6 "2021-07-22T13:03:16Z")

</div>

> [@Strange\_Xue](#):
>
> `n/2* log(σ^2)`

I’m wondering if this is why you think you need `BigFloat` precision? Your `log(σ^2)` expression will spuriously overflow and give `Inf` if `log_σ > log(sqrt(floatmax())) ≈ 355`. However, if you simply rewrite this to `n * log_σ`, as in [my suggested code](https://discourse.julialang.org/t/optim-jl-cant-optimize-function-which-includes-bigfloat-variables/65095/4), the problem goes away and you should get an accurate, finite result even when `log_σ` is large.

(The other term, which divides by `2σ^2`, will simply underflow to `0.0` when `log_σ` is large, so it isn’t a problem.)

---

<div class="post-metadata">

**Author:** ![cgeoga](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cgeoga/32/216186_2.png) [@cgeoga](https://discourse.julialang.org/u/cgeoga)\
**Post date:** [July 22, 2021, 1:12pm UTC](https://discourse.julialang.org/t/optim-jl-cant-optimize-function-which-includes-bigfloat-variables/65095/7 "2021-07-22T13:12:06Z")

</div>

While @jling has definitely answered your actual question, I can’t help but pile on with people who are asking whether it is sensible to use extended precision here: assuming that you care about the uncertainty for your point estimate of \sigma, it probably won’t be so small that the extra decimal places you’d get from using `big` are actual helpful. Like, if your estimate looks like \hat{\sigma} \pm 0.05 or something, then I think using extended precision to get \hat{\sigma} up to 100 decimal points doesn’t actually provide you with any additional information in the sense that the additional digits will change the actual likelihood a negligible amount.

Again, though, not really what you asked, so sorry if this doesn’t actually have anything to do with your real application and you were just looking to make a simple example.