Suppose I define the following problem which computes the maximum likelihood estimates (MLEs) for a regression problem using Optimization.jl:
using Optimization, OptimizationOptimJL, Random, Distributions
β₀, β₁, β₂, β₃, n, σ = -1.0, 1.0, 0.5, 3.0, 300, 0.1
Random.seed!(12301)
x₁ = rand(Uniform(-1, 1), n)
x₂ = rand(Normal(1.0, 0.5), n)
ε = rand(Normal(0.0, σ), n)
y = @. β₀ + β₁ * x₁ + β₂ * x₂ + β₃ * x₁ * x₂ + ε
function loglik(θ, p)
β = @views θ[1:4]
σ² = θ[5]
y, x₁, x₂ = p
ℓℓ = -n / 2 * log(2π * σ²)
for i in 1:length(y)
ℓℓ = ℓℓ - 0.5 / σ² * (y[i] - β[1] - β[2] * x₁[i] - β[3] * x₂[i] - β[4] * x₁[i] * x₂[i])^2
end
return ℓℓ
end
func = OptimizationFunction((u, p) -> -loglik(u, p), Optimization.AutoForwardDiff())
prob = OptimizationProblem(func, ones(5), (y, x₁, x₂); lb = [-Inf,-Inf,-Inf,-Inf,0.0],ub=[Inf,Inf,Inf,Inf,Inf])
solve(prob, NelderMead())
Is there an easy way to extend the existing OptimizationProblem
, prob
, to fix a certain parameter? I’d want to fix say σ
and optimise for the other parameters, over a large range of σ
so it would have to be reasonably efficient. (This is computing a profile likelihood.) This would be done for many parameters, and the computations that will follow it will require the MLE for the full problem, so I do need this full prob
to start with.
I thought about just fixing lb
and ub
for this parameter so that lb = ub = the fixed value of σ
, where I’d use remake
to make this change to prob
, but this seems like it could cause problems. Any ideas?