# Can I "warm start" Optim.jl's BFGS more cleanly?

**URL:** <https://discourse.julialang.org/t/can-i-warm-start-optim-jls-bfgs-more-cleanly/93648>\
**Category:** Optimization (Mathematical)\
**Tags:** optimjl\
**Created:** [January 27, 2023, 4:57pm UTC](https://discourse.julialang.org/t/can-i-warm-start-optim-jls-bfgs-more-cleanly/93648 "2023-01-27T16:57:44Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![lrnv](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lrnv/32/19373_2.png) [@lrnv](https://discourse.julialang.org/u/lrnv)\
**Post date:** [January 27, 2023, 4:57pm UTC](https://discourse.julialang.org/t/can-i-warm-start-optim-jls-bfgs-more-cleanly/93648/1 "2023-01-27T16:57:44Z")

</div>

Hey,

I am solving a series of problems which are in some sense very small modifications from one another.  
At eash step, I modify my loss using the information from the inverse hessian obtained by BFGS, and then i restart BFGS on the new loss. I was wandering if there would be a way to keep the BFGS instance I have (an dnot only give its hessian to a new one) at each step ?

I have something that looks like the following (not an MWE sadly).

```julia

opts = Optim.Options(
    show_trace=true,
    show_every=100,
    allow_f_increases=true,
    allow_outer_f_increases = true,
    iterations=10000
)

my_loss(p,λ) = ...

par = randn(10)
λ = 0
for i in 1:100
    println("------------------------------")
    invH = one(zeros(10,10))
    m = BFGS(initial_invH = x-> invH)
    obj = OnceDifferentiable(p -> my_loss(p,λ),par; autodiff = :forward)
    bfgsstate = Optim.initial_state(m,opts,obj,par) # allows to get back the hessian. 
    res = optimize(obj, par, m, opts, bfgsstate)
    par = Optim.minimizer(res)
    invH = bfgsstate.invH
    λ += some_correction_computed_from(invH)
end

```

This looks wastefull. Is there a way to keep the bfgsstate alltogether and only change the loss in it ? I cannot come up with a good MWE sorry 😕

---

<div class="post-metadata">

**Author:** ![Dan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dan/32/42581_2.png) [@Dan](https://discourse.julialang.org/u/Dan)\
**Post date:** [January 27, 2023, 7:28pm UTC](https://discourse.julialang.org/t/can-i-warm-start-optim-jls-bfgs-more-cleanly/93648/2 "2023-01-27T19:28:31Z")

</div>

Using the NWE (😬), the following seems to be setting the initial state (`p` and `invH`) in my test:

```julia
invH = one(zeros(10,10))
m = BFGS(initial_invH = x -> invH)
par = zeros(10)
obj = OnceDifferentiable(p -> my_loss(p,λ),par; autodiff = :forward)
bfgsstate = Optim.initial_state(m,opts,obj,par)
for i in 1:100
    println("------------------------------")
    res = optimize(obj, par, m, opts, bfgsstate)
    par = Optim.minimizer(res)
    invH .= bfgsstate.invH
    λ += some_correction_computed_from(invH)
    obj = OnceDifferentiable(x -> rosenbrock(x, λ),par; autodiff = :forward)
    Optim.reset!(m, bfgsstate, obj, par)
end

```

`Optim.reset!` is helpful in this case.