# Would it be possible to use multiple CPU cores in Solving Nonlinear Equations?

**URL:** <https://discourse.julialang.org/t/would-it-be-possible-to-use-multiple-cpu-cores-in-solving-nonlinear-equations/113143>\
**Category:** General Usage\
**Tags:** question, nonlinearsolve\
**Created:** [April 18, 2024, 10:49am UTC](https://discourse.julialang.org/t/would-it-be-possible-to-use-multiple-cpu-cores-in-solving-nonlinear-equations/113143 "2024-04-18T10:49:18Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![LINGXIAO\_QIN](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lingxiao_qin/32/206239_2.png) [@LINGXIAO\_QIN](https://discourse.julialang.org/u/LINGXIAO_QIN)\
**Post date:** [April 18, 2024, 10:49am UTC](https://discourse.julialang.org/t/would-it-be-possible-to-use-multiple-cpu-cores-in-solving-nonlinear-equations/113143/1 "2024-04-18T10:49:18Z")

</div>

I am trying to solve a system of nonlinear equations (~25 variables). Currently, I am using NonlinearSolve.jl, it works but I want to expedite the computation. Except for some strategies introduced [here](https://docs.juliahub.com/General/NonlinearSolve/3.10.0/tutorials/code_optimization/) such as using `StaticArray` and specifying the sparse Jacob matrix, I am wondering if it’s feasible to leverage multiple CPU cores to speed up the computation. Currently, the solving process appears to utilize only a single CPU core.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [April 18, 2024, 12:23pm UTC](https://discourse.julialang.org/t/would-it-be-possible-to-use-multiple-cpu-cores-in-solving-nonlinear-equations/113143/2 "2024-04-18T12:23:08Z")

</div>

It will automatically multithread on large equations, but indeed your equation is small.

You can try AutoPolyseterForwardDiff and see if you get a speedup, but that’s dependent on whether your time is generally spent in the Jacobian construction.

Small nonlinear systems are generally not a problem with good parallelization in the algorithms, though it could be a nice research topic.

---

<div class="post-metadata">

**Author:** ![LINGXIAO\_QIN](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lingxiao_qin/32/206239_2.png) [@LINGXIAO\_QIN](https://discourse.julialang.org/u/LINGXIAO_QIN)\
**Post date:** [April 21, 2024, 3:11am UTC](https://discourse.julialang.org/t/would-it-be-possible-to-use-multiple-cpu-cores-in-solving-nonlinear-equations/113143/3 "2024-04-21T03:11:45Z")

</div>

Thanks for your kind suggestion. My equations cannot be automatically differentiated, and thus AutoPolyseterForwardDiff is not a viable choice for me (I am using `autodiff=AutoSparseFiniteDiff()`). I asked this question because in MATLAB, there is a `UseParallel=true` option for `fsolve` to [estimate gradients in parallel](https://www.mathworks.com/help/optim/ug/fsolve.html?searchHighlight=fsolve&s_tid=srchtitle_support_results_1_fsolve). But as you said, “NonlinearSolve.jl will automatically multithread on large equations”.

As a beginner in Julia, I think the slow computation speed is possibly because I don’t write the equations in an efficient way. I’ll look into the [performance tips](https://docs.julialang.org/en/v1/manual/performance-tips/) and see if I can improve my code.

---

<div class="post-metadata">

**Author:** ![avikpal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/avikpal/32/6550_2.png) [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Post date:** [April 21, 2024, 4:04am UTC](https://discourse.julialang.org/t/would-it-be-possible-to-use-multiple-cpu-cores-in-solving-nonlinear-equations/113143/4 "2024-04-21T04:04:52Z")

</div>

For a 25 variable problem, sparsity is most like going to slow down your problem rather that speed it up. Sparsity benefits kick in after around 1000 variables at the minimum [Ill-Conditioned Nonlinear System Work-Precision Diagrams · The SciML Benchmarks](https://docs.sciml.ai/SciMLBenchmarksOutput/stable/NonlinearProblem/bruss/#Scaling-of-Sparsity-Detection-Algorithm)

---

<div class="post-metadata">

**Author:** ![LINGXIAO\_QIN](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lingxiao_qin/32/206239_2.png) [@LINGXIAO\_QIN](https://discourse.julialang.org/u/LINGXIAO_QIN)\
**Post date:** [April 21, 2024, 7:24am UTC](https://discourse.julialang.org/t/would-it-be-possible-to-use-multiple-cpu-cores-in-solving-nonlinear-equations/113143/5 "2024-04-21T07:24:28Z")

</div>

Thank you for your answer. That’s a really informative benchmark. In my 25-variable problem, the jacobian is tridiagonal, so the non-zeros of the matrix is (25+24\*2)/(25\*25)=11.68%. I compared the solving times for `autodiff=AutoFiniteDiff()` and `autodiff=AutoSparseFiniteDiff()`, and found that sparsity does speed up the calculation. However, my objective function seems to suffer from type instability (and maybe many unnecessary allocations), which may contaminate the comparison results. I’ll try to fix my equation constructions.

---

<div class="post-metadata">

**Author:** ![avikpal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/avikpal/32/6550_2.png) [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Post date:** [April 21, 2024, 1:54pm UTC](https://discourse.julialang.org/t/would-it-be-possible-to-use-multiple-cpu-cores-in-solving-nonlinear-equations/113143/6 "2024-04-21T13:54:12Z")

</div>

Ah for Tridiagonal, can you supply a prototype directly as `jac_prototype = <...>` where the prototype is of type Tridiagonal but the entries can be anything?

`AutoSparse...` will generate a sparse matrix, but in case of tridiagonal you can use better factorization and such if the prototype is given.

---

<div class="post-metadata">

**Author:** ![LINGXIAO\_QIN](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lingxiao_qin/32/206239_2.png) [@LINGXIAO\_QIN](https://discourse.julialang.org/u/LINGXIAO_QIN)\
**Post date:** [April 22, 2024, 12:53am UTC](https://discourse.julialang.org/t/would-it-be-possible-to-use-multiple-cpu-cores-in-solving-nonlinear-equations/113143/7 "2024-04-22T00:53:07Z")

</div>

Thanks. By specifing `jac_prototype = ` a [Tridiagonal](https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/#LinearAlgebra.Tridiagonal) and using `autodiff=AutoSparseFiniteDiff()`, an approximately 30% speedup is achieved for my 25-variable problem.