# Newton + LineSearch in Hot Loop: Reduce allocations

**URL:** <https://discourse.julialang.org/t/newton-linesearch-in-hot-loop-reduce-allocations/122585>\
**Category:** Numerics\
**Tags:** newton-raphson, nonlinearsolve\
**Created:** [November 13, 2024, 1:58pm UTC](https://discourse.julialang.org/t/newton-linesearch-in-hot-loop-reduce-allocations/122585 "2024-11-13T13:58:29Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![DanDoe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dandoe/32/52717_2.png) [@DanDoe](https://discourse.julialang.org/u/DanDoe)\
**Post date:** [November 13, 2024, 1:58pm UTC](https://discourse.julialang.org/t/newton-linesearch-in-hot-loop-reduce-allocations/122585/1 "2024-11-13T13:58:29Z")

</div>

I really like the NLsolve.jl package in conjunction with LineSearches.jl - great stuff!

I have a scalar, relatively simple nonlinear equation that I can solve using

```julia
res = nlsolve(gamma -> f(gamma[1], bla, bla, bla),
              gamma -> j(gamma[1], bla, bla, bla),
              [gamma], method = :newton, ftol = 2 * eps(RealT), iterations = 20,
              linesearch = LineSearches.BackTracking(order=3))
gamma = res.zero[1]

```

which works really well.  
The issue is, however, that I get a couple allocations whenever I call this. Is there any way to reduce these by e.g. initalizing the problem once and then updating it later?

The functions `f` and `g` stay constant over time, although their arguments `bla` change over time.

Xref: [Newton + LineSearch in Hot Loop: Reduce allocations · Issue #290 · JuliaNLSolvers/NLsolve.jl · GitHub](https://github.com/JuliaNLSolvers/NLsolve.jl/issues/290#issue-2655636413)

---

<div class="post-metadata">

**Author:** ![avikpal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/avikpal/32/6550_2.png) [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Post date:** [November 13, 2024, 2:20pm UTC](https://discourse.julialang.org/t/newton-linesearch-in-hot-loop-reduce-allocations/122585/2 "2024-11-13T14:20:55Z")

</div>

If you use NonlinearSolve.jl there are a couple of options:

1. Use SimpleNonlinearSolve [Code Optimization for Small Nonlinear Systems in Julia · NonlinearSolve.jl](https://docs.sciml.ai/NonlinearSolve/stable/tutorials/code_optimization/#Further-Optimizations-for-Small-Nonlinear-Solves-with-Static-Arrays-and-SimpleNonlinearSolve) that will pretty much outperform most solvers for scalar/static arrays problems
2. Use the caching interface of `cache = init(....)` and inside the loop first do `reinit!(cache; p = <bla, bla, bla>)` and then `solve!(cache)`

---

<div class="post-metadata">

**Author:** ![DanDoe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dandoe/32/52717_2.png) [@DanDoe](https://discourse.julialang.org/u/DanDoe)\
**Post date:** [November 13, 2024, 2:30pm UTC](https://discourse.julialang.org/t/newton-linesearch-in-hot-loop-reduce-allocations/122585/3 "2024-11-13T14:30:03Z")

</div>

Thanks for your swift response! Does `NonlinearSolve.jl` also support LineSearches? These are quite vital to speed up the Newton procedure.

---

<div class="post-metadata">

**Author:** ![avikpal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/avikpal/32/6550_2.png) [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Post date:** [November 13, 2024, 2:36pm UTC](https://discourse.julialang.org/t/newton-linesearch-in-hot-loop-reduce-allocations/122585/4 "2024-11-13T14:36:57Z")

</div>

Yes. You can use any LineSearches algorithm via [LineSearches.jl · LineSearch.jl](https://sciml.github.io/LineSearch.jl/dev/api/line_searches/). There’s also [Native Functionalities · LineSearch.jl](https://sciml.github.io/LineSearch.jl/dev/api/native/)

---

<div class="post-metadata">

**Author:** ![DanDoe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dandoe/32/52717_2.png) [@DanDoe](https://discourse.julialang.org/u/DanDoe)\
**Post date:** [November 13, 2024, 4:05pm UTC](https://discourse.julialang.org/t/newton-linesearch-in-hot-loop-reduce-allocations/122585/5 "2024-11-13T16:05:39Z")

</div>

Okay, one more thing: It seems to be not as trivial to set the Jacobian by hand (I cannot resort to AD). I found [this post](https://discourse.julialang.org/t/specifying-analytical-jacobian-in-nonlinearproblem/112932) but unfortunately `NonlinearSolve.jl` still tries to compute the Jacobian using AD.

---

<div class="post-metadata">

**Author:** ![avikpal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/avikpal/32/6550_2.png) [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Post date:** [November 13, 2024, 4:06pm UTC](https://discourse.julialang.org/t/newton-linesearch-in-hot-loop-reduce-allocations/122585/6 "2024-11-13T16:06:27Z")

</div>

`NonlinearFunction(f; jac = ...)`

---

<div class="post-metadata">

**Author:** ![DanDoe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dandoe/32/52717_2.png) [@DanDoe](https://discourse.julialang.org/u/DanDoe)\
**Post date:** [November 14, 2024, 8:08am UTC](https://discourse.julialang.org/t/newton-linesearch-in-hot-loop-reduce-allocations/122585/7 "2024-11-14T08:08:18Z")

</div>

Xref: [NonlinearSolve uses AD Jacobian even when I specify it analytically - #3 by DanDoe](https://discourse.julialang.org/t/nonlinearsolve-uses-ad-jacobian-even-when-i-specify-it-analytically/122608/3)