# Optim with occasionally non-finite values

**URL:** <https://discourse.julialang.org/t/optim-with-occasionally-non-finite-values/10174>\
**Category:** Optimization (Mathematical)\
**Tags:** question\
**Created:** [April 5, 2018, 1:27pm UTC](https://discourse.julialang.org/t/optim-with-occasionally-non-finite-values/10174 "2018-04-05T13:27:30Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [April 5, 2018, 1:27pm UTC](https://discourse.julialang.org/t/optim-with-occasionally-non-finite-values/10174/1 "2018-04-05T13:27:31Z")

</div>

I am maximizing a somewhat difficult likelihood function, for which the values are occasionally non-finite. I run into stack traces like

```julia
ERROR: AssertionError: isfinite(phid) && isfinite(gphi)
Stacktrace:
 [1] bisect!(::Optim.ManifoldObjective{NLSolversBase.OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1},Val{false}}}, ::Array{Float64,1}, ::Array{Float64,1}, ::Array{Float64,1}, ::LineSearches.LineSearchResults{Float64}, ::Int64, ::Int64, ::Float64, ::Int64) at /home/tamas/.julia/v0.6/LineSearches/src/hagerzhang.jl:473
 [2] _hagerzhang!(::Optim.ManifoldObjective{NLSolversBase.OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1},Val{false}}}, ::Array{Float64,1}, ::Array{Float64,1}, ::Array{Float64,1}, ::LineSearches.LineSearchResults{Float64}, ::Float64, ::Bool, ::Float64, ::Float64, ::Float64, ::Float64, ::Float64, ::Float64, ::Int64, ::Float64, ::Int64) at /home/tamas/.julia/v0.6/LineSearches/src/hagerzhang.jl:188
 [3] perform_linesearch!(::Optim.LBFGSState{Array{Float64,1},Array{Array{Float64,1},1},Array{Array{Float64,1},1},Float64,Array{Float64,1}}, ::Optim.LBFGS{Void,LineSearches.InitialStatic{Float64},LineSearches.HagerZhang{Float64},Optim.##76#78}, ::Optim.ManifoldObjective{NLSolversBase.OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1},Val{false}}}) at /home/tamas/.julia/v0.6/Optim/src/utilities/perform_linesearch.jl:47
 [4] update_state!(::NLSolversBase.OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1},Val{false}}, ::Optim.LBFGSState{Array{Float64,1},Array{Array{Float64,1},1},Array{Array{Float64,1},1},Float64,Array{Float64,1}}, ::Optim.LBFGS{Void,LineSearches.InitialStatic{Float64},LineSearches.HagerZhang{Float64},Optim.##76#78}) at /home/tamas/.julia/v0.6/Optim/src/multivariate/solvers/first_order/l_bfgs.jl:198
 [5] optimize(::NLSolversBase.OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1},Val{false}}, ::Array{Float64,1}, ::Optim.LBFGS{Void,LineSearches.InitialStatic{Float64},LineSearches.HagerZhang{Float64},Optim.##76#78}, ::Optim.Options{Float64,Void}, ::Optim.LBFGSState{Array{Float64,1},Array{Array{Float64,1},1},Array{Array{Float64,1},1},Float64,Array{Float64,1}}) at /home/tamas/.julia/v0.6/Optim/src/multivariate/optimize/optimize.jl:51
 [6] optimize(::NLSolversBase.OnceDifferentiable{Float64,Array{Float64,1},Array{Float64,1},Val{false}}, ::Array{Float64,1}, ::Optim.LBFGS{Void,LineSearches.InitialStatic{Float64},LineSearches.HagerZhang{Float64},Optim.##76#78}) at /home/tamas/.julia/v0.6/Optim/src/multivariate/optimize/optimize.jl:25

```

Can I somehow convince `Optim.optimize` to just gracefully ignore these proposed values? Using `BGFS()` with `autodiff = :forward`.

---

<div class="post-metadata">

**Author:** ![anriseth](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/anriseth/32/3106_2.png) [@anriseth](https://discourse.julialang.org/u/anriseth)\
**Post date:** [April 5, 2018, 4:44pm UTC](https://discourse.julialang.org/t/optim-with-occasionally-non-finite-values/10174/2 "2018-04-05T16:44:33Z")

</div>

I see the problem happens inside the `HagerZhang` linesearch. You may have more luck with one of the other line search algorithms. Try them in the following order.

- `BackTracking(order=2)`
- `BackTracking()`
- `MoreThuente()`
- `Static()`

Static is the simplest of the line searches, as it just takes a fixed step size. This can work well if your outer optimizer (BFGS or whatever) have sensible scalings of the step direction.

I would also suggest trying `LBFGS()`, which after some tweaks last year performs quite well.

For example,

```julia
using LineSearches, Optim
df = OnceDifferentiable(...)
res = optimize(df, LBFGS(linesearch = BackTracking(order=2)))

```

EDIT: I see you mention “maximizing”, so I thought I’d just double check that you have remembered to turn the problem into a minimization problem before passing it to Optim?