# Controls the number of LBFGS iterations with optim.jl

**URL:** <https://discourse.julialang.org/t/controls-the-number-of-lbfgs-iterations-with-optim-jl/70455>\
**Category:** Performance\
**Tags:** package, optim, optimization\
**Created:** [October 27, 2021, 7:29am UTC](https://discourse.julialang.org/t/controls-the-number-of-lbfgs-iterations-with-optim-jl/70455 "2021-10-27T07:29:39Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![F-YF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/f-yf/32/17363_2.png) [@F-YF](https://discourse.julialang.org/u/F-YF)\
**Post date:** [October 27, 2021, 7:29am UTC](https://discourse.julialang.org/t/controls-the-number-of-lbfgs-iterations-with-optim-jl/70455/1 "2021-10-27T07:29:39Z")

</div>

I want to use the LBFGS() method in optim package to optimize. Usually LBFGS will do many linearSearch, but I only want it to do one iteration. How to do that?

[LBFGS](https://github.com/JuliaNLSolvers/Optim.jl/blob/master/src/multivariate/solvers/first_order/l_bfgs.jl#L2)

---

<div class="post-metadata">

**Author:** ![mcreel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcreel/32/30088_2.png) [@mcreel](https://discourse.julialang.org/u/mcreel)\
**Post date:** [October 27, 2021, 12:40pm UTC](https://discourse.julialang.org/t/controls-the-number-of-lbfgs-iterations-with-optim-jl/70455/2 "2021-10-27T12:40:07Z")

</div>

Just set the iterations option to 1, for example:  
result = optimize(rosenbrock, zeros(2), LBFGS(), Optim.Options(iterations=1))

---

<div class="post-metadata">

**Author:** ![F-YF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/f-yf/32/17363_2.png) [@F-YF](https://discourse.julialang.org/u/F-YF)\
**Post date:** [October 27, 2021, 1:11pm UTC](https://discourse.julialang.org/t/controls-the-number-of-lbfgs-iterations-with-optim-jl/70455/3 "2021-10-27T13:11:03Z")

</div>

Hm, This is not what I mean. Maybe I didn’t go into enough detail  
This is a minimal working example

```julia
using Optim
function f(x)
    temp=(1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2
    println("f=",temp)
    return temp
end
result = optimize(f, [1.0 1.0], LBFGS(), Optim.Options(show_trace=true,extended_trace=true,iterations=1))

```

The output is there:

```julia
f=1.4704168796019891e-8
f=1.47039911610062e-8
f=3.666852862404853e-9
f=3.6668528625393112e-9
f=0.0
Iter Function value Gradient norm 
     0 0.000000e+00 1.466736e-08
 * Current step size: 1.0
 * time: 0.0
 * g(x): [1.4667356107373247e-8 -1.1102239280930583e-14]
 * x: [1.0 1.0]
f=1.4633022753087148e-8
f=1.4775308449140644e-8
f=3.702466146048789e-9
f=3.6314121143532064e-9
f=8.626772924165988e-14
f=1.470407997780277e-8
f=1.4704079977879413e-8
f=3.666897160566489e-9
f=3.6668085646459185e-9
f=1.3412189577348004e-19
f=1.4704124387146717e-8
f=1.4704035569409278e-8
f=3.6668750114521403e-9
f=3.666830713559084e-9
f=3.353047394337001e-20
f=1.4704146591574923e-8
f=1.4704013365333815e-8
f=3.666863936920114e-9
f=3.666841788040815e-9
f=8.382618485842503e-21
f=1.4704157694334489e-8
f=1.4704002263167913e-8
f=3.666858399660388e-9
f=3.666847325287967e-9
f=2.0956546214606257e-21
f=1.4704163245715843e-8
f=1.4703996711816946e-8
f=3.6668556308976383e-9
f=3.6668500940475728e-9
f=5.238627689649373e-22
f=1.470416602032856e-8
f=1.470399393668103e-8
f=3.666854246785573e-9
f=3.666851478158853e-9
f=1.3099113605918222e-22
f=1.4704167408174191e-8
f=1.4703992548843586e-8
f=3.66685355459518e-9
f=3.666852170349049e-9
f=3.2747784014795554e-23
f=1.4704168102636214e-8
f=1.4703991854924885e-8
f=3.6668532085000085e-9
f=3.666852516444172e-9
f=8.186946003698888e-24
f=1.4704168449867228e-8
f=1.4703991507695954e-8
f=3.6668530353179703e-9
f=3.6668526896261973e-9
f=2.0435571048983057e-24
f=1.4704168622404382e-8
f=1.4703991334620665e-8
f=3.666852948995869e-9
f=3.6668527759482952e-9
f=5.124795920761108e-25
f=1.4704168709751314e-8
f=1.4703991247813434e-8
f=3.6668529057003606e-9
f=3.666852819243803e-9
f=1.281198980190277e-25
f=1.4704168752885602e-8
f=1.4703991204140229e-8
f=3.666852883918149e-9
f=3.6668528410260152e-9
f=3.1633322299362573e-26
f=1.4704168774452748e-8
f=1.4703991182573215e-8
f=3.6668528731615006e-9
f=3.666852851782663e-9
f=7.908330574840643e-27
f=1.470416878523632e-8
f=1.4703991171789708e-8
f=3.666852867783177e-9
f=3.6668528571609866e-9
f=1.977082643710161e-27
f=1.4704168790628105e-8
f=1.4703991166397954e-8
f=3.6668528650940147e-9
f=3.666852859850149e-9
f=4.942706609275402e-28
f=1.4704168793863177e-8
f=1.4703991163702078e-8
f=3.666852863749434e-9
f=3.6668528611947296e-9
f=1.2356766523188505e-28
f=1.4704168794941536e-8
f=1.4703991162623728e-8
f=3.6668528632116014e-9
f=3.6668528617325624e-9
f=4.448435948347862e-29
f=1.4704168796019891e-8
f=1.4703991161545379e-8
f=3.6668528626737686e-9
f=3.666852862270395e-9
f=4.942706609275402e-30
f=1.4704168796019891e-8
f=1.47039911610062e-8
f=3.666852862404853e-9
f=3.6668528625393112e-9
f=0.0
f=1.4704168796019891e-8
f=1.4703991161545379e-8
f=3.6668528626737686e-9
f=3.666852862270395e-9
f=4.942706609275402e-30
f=1.4704168796019891e-8
f=1.47039911610062e-8
f=3.666852862404853e-9
f=3.6668528625393112e-9
f=0.0
f=1.4704168796019891e-8
f=1.4703991161545379e-8
f=3.6668528626737686e-9
f=3.666852862270395e-9
f=4.942706609275402e-30
f=1.4704168796019891e-8
f=1.47039911610062e-8
f=3.666852862404853e-9
f=3.6668528625393112e-9
f=0.0
f=1.4704168796019891e-8
f=1.4703991161545379e-8
f=3.6668528626737686e-9
f=3.666852862270395e-9
f=4.942706609275402e-30
f=1.4704168796019891e-8
f=1.47039911610062e-8
f=3.666852862404853e-9
f=3.6668528625393112e-9
f=0.0
f=1.4704168796019891e-8
f=1.4703991161545379e-8
f=3.6668528626737686e-9
f=3.666852862270395e-9
f=4.942706609275402e-30
f=1.4704168796019891e-8
f=1.47039911610062e-8
f=3.666852862404853e-9
f=3.6668528625393112e-9
f=0.0
f=1.4704168796019891e-8
f=1.4703991161545379e-8
f=3.6668528626737686e-9
f=3.666852862270395e-9
f=4.942706609275402e-30
f=1.4704168796019891e-8
f=1.47039911610062e-8
f=3.666852862404853e-9
f=3.6668528625393112e-9
f=0.0
f=1.4704168796019891e-8
f=1.4703991161545379e-8
f=3.6668528626737686e-9
f=3.666852862270395e-9
f=4.942706609275402e-30
f=1.4704168796019891e-8
f=1.47039911610062e-8
f=3.666852862404853e-9
f=3.6668528625393112e-9
f=0.0
f=1.4704168796019891e-8
f=1.4703991161545379e-8
f=3.6668528626737686e-9
f=3.666852862270395e-9
f=4.942706609275402e-30
f=1.4704168796019891e-8
f=1.47039911610062e-8
f=3.666852862404853e-9
f=3.6668528625393112e-9
f=0.0
f=1.4704168796019891e-8
f=1.4703991161545379e-8
f=3.6668528626737686e-9
f=3.666852862270395e-9
f=4.942706609275402e-30
f=1.4704168796019891e-8
f=1.47039911610062e-8
f=3.666852862404853e-9
f=3.6668528625393112e-9
f=0.0
f=1.4704168796019891e-8
f=1.4703991161545379e-8
f=3.6668528626737686e-9
f=3.666852862270395e-9
f=4.942706609275402e-30
f=1.4704168796019891e-8
f=1.47039911610062e-8
f=3.666852862404853e-9
f=3.6668528625393112e-9
f=0.0
f=1.4704168796019891e-8
f=1.4703991161545379e-8
f=3.6668528626737686e-9
f=3.666852862270395e-9
f=4.942706609275402e-30
f=1.4704168796019891e-8
f=1.47039911610062e-8
f=3.666852862404853e-9
f=3.6668528625393112e-9
f=0.0
f=1.4704168796019891e-8
f=1.4703991161545379e-8
f=3.6668528626737686e-9
f=3.666852862270395e-9
f=4.942706609275402e-30
f=1.4704168796019891e-8
f=1.47039911610062e-8
f=3.666852862404853e-9
f=3.6668528625393112e-9
f=0.0
f=1.4704168796019891e-8
f=1.4703991161545379e-8
f=3.6668528626737686e-9
f=3.666852862270395e-9
f=4.942706609275402e-30
f=1.4704168796019891e-8
f=1.47039911610062e-8
f=3.666852862404853e-9
f=3.6668528625393112e-9
f=0.0
f=1.4704168796019891e-8
f=1.4703991161545379e-8
f=3.6668528626737686e-9
f=3.666852862270395e-9
f=4.942706609275402e-30
f=1.4704168796019891e-8
f=1.47039911610062e-8
f=3.666852862404853e-9
f=3.6668528625393112e-9
f=0.0
f=1.4704168796019891e-8
f=1.4703991161545379e-8
f=3.6668528626737686e-9
f=3.666852862270395e-9
f=4.942706609275402e-30
f=1.4704168796019891e-8
f=1.47039911610062e-8
f=3.666852862404853e-9
f=3.6668528625393112e-9
f=0.0
     1 0.000000e+00 1.466736e-08
 * Current step size: 3.784673312960098e-9
 * time: 1.1170001029968262
 * g(x): [1.4667356107373247e-8 -1.1102239280930583e-14]
 * x: [1.0 1.0]
 * Status: success

 * Candidate solution
    Final objective value: 0.000000e+00

 * Found with
    Algorithm: L-BFGS

 * Convergence measures
    |x - x'| = 0.00e+00 ≤ 0.0e+00
    |x - x'|/|x'| = 0.00e+00 ≤ 0.0e+00
    |f(x) - f(x')| = 0.00e+00 ≤ 0.0e+00
    |f(x) - f(x')|/|f(x')| = NaN ≰ 0.0e+00
    |g(x)| = 1.47e-08 ≰ 1.0e-08

 * Work counters
    Seconds run: 1 (vs limit Inf)
    Iterations: 1
    f(x) calls: 49
    ∇f(x) calls: 49

```

From the above results, we can see that the number of optimization iterations is 1, but many linear iterations are done .

When my function is very complex, multiple linear searches will call the function many times, but the optimization effect is not significant . So the number of iterations I want to control is the number of iterations of linear search, and the output that I want is going to look like this :

```julia
Iter Function value Gradient norm 
     0 0.000000e+00 1.466736e-08
 * Current step size: 1.0
 * time: 0.0
 * g(x): [1.4667356107373247e-8 -1.1102239280930583e-14]
 * x: [1.0 1.0]
f=1.4633022753087148e-8
 1 0.000000e+00 1.466736e-08
 * Current step size: 3.784673312960098e-9
 * time: 1.1170001029968262
 * g(x): [1.4667356107373247e-8 -1.1102239280930583e-14]
 * x: [1.0 1.0]

```

---

<div class="post-metadata">

**Author:** ![IlyaOrson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ilyaorson/32/1681_2.png) [@IlyaOrson](https://discourse.julialang.org/u/IlyaOrson)\
**Post date:** [October 27, 2021, 2:43pm UTC](https://discourse.julialang.org/t/controls-the-number-of-lbfgs-iterations-with-optim-jl/70455/4 "2021-10-27T14:43:38Z")

</div>

Linesearch routines are in a separate package `LineSearches.jl`, LBFGS takes them as a keyword.

```julia
linesearch = LineSearches.BackTracking(iterations=10)
LBFGS(; linesearch=linesearch)

```

---

<div class="post-metadata">

**Author:** ![pkofod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pkofod/32/2179_2.png) [@pkofod](https://discourse.julialang.org/u/pkofod)\
**Post date:** [October 27, 2021, 8:24pm UTC](https://discourse.julialang.org/t/controls-the-number-of-lbfgs-iterations-with-optim-jl/70455/5 "2021-10-27T20:24:03Z")

</div>

Yes.

So you can do as @IlyaOrson says above and set it to 1. Or use `LineSearches.Static()`. I don’t think you want to, but you can 🙂

> [@F-YF](#):
>
> When my function is very complex, multiple linear searches will call the function many times, but the optimization effect is not significant . So the number of iterations I want to control is the number of iterations of linear search, and the output that I want is going to look like this :

This part I don’t understand. If you set it to 1, you you probably get “Current step size” of 1 or 1/2 depending on what you do.

---

<div class="post-metadata">

**Author:** ![F-YF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/f-yf/32/17363_2.png) [@F-YF](https://discourse.julialang.org/u/F-YF)\
**Post date:** [October 28, 2021, 2:51am UTC](https://discourse.julialang.org/t/controls-the-number-of-lbfgs-iterations-with-optim-jl/70455/6 "2021-10-28T02:51:38Z")

</div>

> [@IlyaOrson](#):
>
> ```julia
> linesearch = LineSearches.BackTracking(iterations=10)
> LBFGS(; linesearch=linesearch)
> 
> ```

> [@pkofod](#):
>
> you can do as @IlyaOrson says above and set it to 1. Or use `LineSearches.Static()`

Great. Thank you for your reply. It’s very helpful for me