# Simplest way to convert a program for parallel (multithreaded) runs on multiple servers/cores?

**URL:** <https://discourse.julialang.org/t/simplest-way-to-convert-a-program-for-parallel-multithreaded-runs-on-multiple-servers-cores/55363>\
**Category:** Julia at Scale\
**Created:** [February 16, 2021, 12:08am UTC](https://discourse.julialang.org/t/simplest-way-to-convert-a-program-for-parallel-multithreaded-runs-on-multiple-servers-cores/55363 "2021-02-16T00:08:47Z")\
**Posts on this page:** 7\
**Page:** 2

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [February 18, 2021, 4:06pm UTC](https://discourse.julialang.org/t/simplest-way-to-convert-a-program-for-parallel-multithreaded-runs-on-multiple-servers-cores/55363/22 "2021-02-18T16:06:07Z")

</div>

You just turn `sparse=true` on in the generation call.

---

<div class="post-metadata">

**Author:** ![CosmoProf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cosmoprof/32/15516_2.png) [@CosmoProf](https://discourse.julialang.org/u/CosmoProf)\
**Post date:** [February 18, 2021, 9:40pm UTC](https://discourse.julialang.org/t/simplest-way-to-convert-a-program-for-parallel-multithreaded-runs-on-multiple-servers-cores/55363/23 "2021-02-18T21:40:59Z")

</div>

With `CVODE_BDF` finally working, using that instead of `Rodas5`, the runtime of my main program for `Npix = 1601` was reduced from 8 hours to 2 hours! Wow!

---

<div class="post-metadata">

**Author:** ![CosmoProf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cosmoprof/32/15516_2.png) [@CosmoProf](https://discourse.julialang.org/u/CosmoProf)\
**Post date:** [February 19, 2021, 8:48am UTC](https://discourse.julialang.org/t/simplest-way-to-convert-a-program-for-parallel-multithreaded-runs-on-multiple-servers-cores/55363/24 "2021-02-19T08:48:29Z")

</div>

One last question, if I may… since I am running on a system with multiple cores, is there a straightforward way to make the command `DifferentialEquations.solve` (from my sample program above) run in a distributed way on the multiple cores, for savings of time and memory? (BTW, I run the program from the IDE, not from command line.)

(I tried reading through “[Multi-processing and Distributed Computing · The Julia Language](https://docs.julialang.org/en/v1/manual/distributed-computing/)”, but I’m afraid that it did not illuminate me.)

Thanks again…!

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [February 19, 2021, 11:04am UTC](https://discourse.julialang.org/t/simplest-way-to-convert-a-program-for-parallel-multithreaded-runs-on-multiple-servers-cores/55363/25 "2021-02-19T11:04:03Z")

</div>

Concerning your code, you can accelerate that inner loop with (start julia with `julia -t N` where `N` is the number of cores you want to use):

```julia
    Threads.@threads for i in 2:(Npts-1)
        du[i,2] = u[i,1]
        du[i,1] = (-k*u[i,2]) + ((u[(i+1),2]-(2*u[i,2])+u[(i-1),2])/(DX^2))
    end

```

If that makes a difference for overall peformance, you might want to rewrite the loop to actually take advantage of multi-threading, but I would test that first.

you can also (with a single core), use:

`using LoopVectorization`

and change that loop to:

```julia
    @avx for i in 2:(Npts-1)
        du[i,2] = u[i,1]
        du[i,1] = (-k*u[i,2]) + ((u[(i+1),2]-(2*u[i,2])+u[(i-1),2])/(DX^2))
    end

```

that provides some additional speedup.

But those things will only make any difference if the computation of that loop is important for the overall performance, which I am not sure (I could not run your example).

---

<div class="post-metadata">

**Author:** ![rveltz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rveltz/32/2707_2.png) [@rveltz](https://discourse.julialang.org/u/rveltz)\
**Post date:** [February 19, 2021, 11:30am UTC](https://discourse.julialang.org/t/simplest-way-to-convert-a-program-for-parallel-multithreaded-runs-on-multiple-servers-cores/55363/26 "2021-02-19T11:30:01Z")

</div>

The bottleneck is very likely the computation of the jacobian. Also, you can pass options to CVODE\_BDF like dense, banded if I recall

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [February 19, 2021, 1:56pm UTC](https://discourse.julialang.org/t/simplest-way-to-convert-a-program-for-parallel-multithreaded-runs-on-multiple-servers-cores/55363/27 "2021-02-19T13:56:26Z")

</div>

The simplest thing to try is `CVODE_BDF(linear_solver=:GMRES)` which would be matrix-free. Without a preconditioner 🤷‍♂️ you might get a speedup you might not, but it’s at least easy to check.

---

<div class="post-metadata">

**Author:** ![CosmoProf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cosmoprof/32/15516_2.png) [@CosmoProf](https://discourse.julialang.org/u/CosmoProf)\
**Post date:** [February 22, 2021, 2:00am UTC](https://discourse.julialang.org/t/simplest-way-to-convert-a-program-for-parallel-multithreaded-runs-on-multiple-servers-cores/55363/28 "2021-02-22T02:00:25Z")

</div>

Hi again folks, I have implemented some of these changes, and the results are of the “impossibly too good to be true” variety.

For my real (very large) program, solving the coupled ODE system originally took over 3 days… using `Rodas5()`, the whole program with everything took over 82 hours!

Using `CVODE_BDF()` the program took only 2.5 hours (148 minutes).

Then, using `CVODE_BDF(linear_solver=:GMRES)` took less than 1 hour (51 minutes), and most of that time wasn’t even doing ODE solving!

I was enormously skeptical about this impossible amount of speedup, so I compared the ODE solutions… and the biggest difference in any value between the different solvers was less than 3E-10.

The solver doesn’t even seem to scale with the number of pixels anymore! Running my simpler program shown above with `CVODE_BDF(linear_solver=:GMRES)` – noting that it must include the corrected line, `"maxiters = Int(1e7)"`, to run successfully – going from `Npix = 801` to `1601` to `3201` only increased the `DifferentialEquations.solve` command runtime from 4.87 to 5.87 to 6.52 seconds!

I’m not sure how any of this is possible, but I’m glad it is! Thanks for all of your suggestions… I’m still implementing more of them as necessary…

[Previous page](https://discourse.julialang.org/t/simplest-way-to-convert-a-program-for-parallel-multithreaded-runs-on-multiple-servers-cores/55363.md?page=1)