Optim.jl can't optimize function which includes BigFloat variables

There is no point in converting a single term like σ to BigFloat if the rest of the computations are done in double precision. e.g. log(2π) is computed in double precision, and that will generally limit your accuracy to double precision even if parts of the computation are BigFloat.

(Aside: don’t use global variables like n in your functions if you can help it. In this case, you should get n = length(Y) from the arguments.)

In general, you should try to write code that works for any number type based on the types of the arguments. This can be a little tricky but the result is much more flexible code. For example:

function Log_Likelihood2(X, Y, β, log_σ)
    n = length(Y)
    σ = exp(log_σ)
    llike = n * log_σ + sum((Y .- X * β).^2) / 2σ^2
    return llike + n * log(2*oftype(llike, π)) / 2
end

Not that it uses oftype(llike, π) to compute π in the same precision as llike, and rearranges the calculation to avoid n/2 (Int/Int uses double precision, though technically dividing integers by 2 can be done exactly it still changes the precision).

Now you can compute in whatever precision you want — but you should probably pass all of the arguments in the same precision, as otherwise the accuracy will be limited by the least-precise argument.

This is also important for automatic differentiation, since that works by passing “just another number type” (ForwardDiff.Dual) that keeps track of derivatives.

2 Likes