Optim with occasionally non-finite values

Specific Domains Optimization (Mathematical)
1

I am maximizing a somewhat difficult likelihood function, for which the values are occasionally non-finite. I run into stack traces like

ERROR: AssertionError: isfinite(phid) && isfinite(gphi)
Stacktrace:
 [1] bisect!(::Optim.ManifoldObjective{NLSolversBase.OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1},Val{false}}}, ::Array{Float64,1}, ::Array{Float64,1}, ::Array{Float64,1}, ::LineSearches.LineSearchResults{Float64}, ::Int64, ::Int64, ::Float64, ::Int64) at /home/tamas/.julia/v0.6/LineSearches/src/hagerzhang.jl:473
 [2] _hagerzhang!(::Optim.ManifoldObjective{NLSolversBase.OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1},Val{false}}}, ::Array{Float64,1}, ::Array{Float64,1}, ::Array{Float64,1}, ::LineSearches.LineSearchResults{Float64}, ::Float64, ::Bool, ::Float64, ::Float64, ::Float64, ::Float64, ::Float64, ::Float64, ::Int64, ::Float64, ::Int64) at /home/tamas/.julia/v0.6/LineSearches/src/hagerzhang.jl:188
 [3] perform_linesearch!(::Optim.LBFGSState{Array{Float64,1},Array{Array{Float64,1},1},Array{Array{Float64,1},1},Float64,Array{Float64,1}}, ::Optim.LBFGS{Void,LineSearches.InitialStatic{Float64},LineSearches.HagerZhang{Float64},Optim.##76#78}, ::Optim.ManifoldObjective{NLSolversBase.OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1},Val{false}}}) at /home/tamas/.julia/v0.6/Optim/src/utilities/perform_linesearch.jl:47
 [4] update_state!(::NLSolversBase.OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1},Val{false}}, ::Optim.LBFGSState{Array{Float64,1},Array{Array{Float64,1},1},Array{Array{Float64,1},1},Float64,Array{Float64,1}}, ::Optim.LBFGS{Void,LineSearches.InitialStatic{Float64},LineSearches.HagerZhang{Float64},Optim.##76#78}) at /home/tamas/.julia/v0.6/Optim/src/multivariate/solvers/first_order/l_bfgs.jl:198
 [5] optimize(::NLSolversBase.OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1},Val{false}}, ::Array{Float64,1}, ::Optim.LBFGS{Void,LineSearches.InitialStatic{Float64},LineSearches.HagerZhang{Float64},Optim.##76#78}, ::Optim.Options{Float64,Void}, ::Optim.LBFGSState{Array{Float64,1},Array{Array{Float64,1},1},Array{Array{Float64,1},1},Float64,Array{Float64,1}}) at /home/tamas/.julia/v0.6/Optim/src/multivariate/optimize/optimize.jl:51
 [6] optimize(::NLSolversBase.OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1},Val{false}}, ::Array{Float64,1}, ::Optim.LBFGS{Void,LineSearches.InitialStatic{Float64},LineSearches.HagerZhang{Float64},Optim.##76#78}) at /home/tamas/.julia/v0.6/Optim/src/multivariate/optimize/optimize.jl:25

Can I somehow convince Optim.optimize to just gracefully ignore these proposed values? Using BGFS() with autodiff = :forward.

2

I see the problem happens inside the HagerZhang linesearch. You may have more luck with one of the other line search algorithms. Try them in the following order.

  • BackTracking(order=2)
  • BackTracking()
  • MoreThuente()
  • Static()

Static is the simplest of the line searches, as it just takes a fixed step size. This can work well if your outer optimizer (BFGS or whatever) have sensible scalings of the step direction.

I would also suggest trying LBFGS(), which after some tweaks last year performs quite well.

For example,

using LineSearches, Optim
df = OnceDifferentiable(...)
res = optimize(df, LBFGS(linesearch = BackTracking(order=2)))

EDIT: I see you mention “maximizing”, so I thought I’d just double check that you have remembered to turn the problem into a minimization problem before passing it to Optim?

4 Likes