# Solving ODEs in parallel on a supercomputer

**URL:** <https://discourse.julialang.org/t/solving-odes-in-parallel-on-a-supercomputer/95637>\
**Category:** Modelling & Simulations\
**Created:** [March 6, 2023, 10:11pm UTC](https://discourse.julialang.org/t/solving-odes-in-parallel-on-a-supercomputer/95637 "2023-03-06T22:11:28Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nick1](https://avatars.discourse-cdn.com/v4/letter/n/47e85d/32.png) [@Nick1](https://discourse.julialang.org/u/Nick1)\
**Post date:** [March 6, 2023, 10:11pm UTC](https://discourse.julialang.org/t/solving-odes-in-parallel-on-a-supercomputer/95637/1 "2023-03-06T22:11:28Z")

</div>

Hi everyone,

I would like to solve ODEs in parallel on a supercomputer. I can use one or more nodes and multiple cores per node. There is also the option for node sharing. The job scheduler for the supercomputer is slurm. Which algorithm is the most appropriate for each case (single node, multi-node, multi-core, etc)?

List of algorithms: [Parallel Ensemble Simulations · DifferentialEquations.jl](https://docs.sciml.ai/DiffEqDocs/dev/features/ensemble/)

---

<div class="post-metadata">

**Author:** ![liamfdoherty](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/liamfdoherty/32/24516_2.png) [@liamfdoherty](https://discourse.julialang.org/u/liamfdoherty)\
**Post date:** [March 6, 2023, 10:21pm UTC](https://discourse.julialang.org/t/solving-odes-in-parallel-on-a-supercomputer/95637/2 "2023-03-06T22:21:44Z")

</div>

Which “algorithm” do you mean? The type of parallelism, or the ODE solver? The ODE solver shouldn’t matter, I don’t think, in the sense that the fastest serial algorithm should also be able to scale to parallel machines just fine. If you need a stiff solver to solve one instance of the problem, just use a stiff solver on many cores.

If you mean which parallelization method to choose, the descriptions are further down the page [here](https://docs.sciml.ai/DiffEqDocs/dev/features/ensemble/#EnsembleAlgorithms). If you have access to GPUs on your cluster, use `EnsembleGPUArray`. If not, and your system is easy, there will likely be no problem using the default `EnsembleThreads`, and if your problem is hard, you may consider `EnsembleDistributed` instead.

Another note:  
I’m not 100% sure that it’s flawless because I haven’t used it in a while and the last commit was a couple months ago, but `ClusterManagers.jl` offers easy-to-use functionality for interfacing with supercomputers, and includes support for the Slurm scheduler.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [March 6, 2023, 10:21pm UTC](https://discourse.julialang.org/t/solving-odes-in-parallel-on-a-supercomputer/95637/3 "2023-03-06T22:21:49Z")

</div>

How many ODEs and how big are they?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 7, 2023, 8:37am UTC](https://discourse.julialang.org/t/solving-odes-in-parallel-on-a-supercomputer/95637/4 "2023-03-07T08:37:50Z")

</div>

> [@liamfdoherty](#):
>
> If you have access to GPUs on your cluster, use `EnsembleGPUArray`.

Note that the newer EnsembleGPUKernel is much faster.

> **[DiffEqGPU: Massively Data-Parallel GPU Solving of ODEs · DiffEqGPU.jl](https://docs.sciml.ai/DiffEqGPU/stable/)**
>
> Documentation for DiffEqGPU.jl.

Though only has specific algorithms.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 8, 2023, 1:48pm UTC](https://discourse.julialang.org/t/solving-odes-in-parallel-on-a-supercomputer/95637/5 "2023-03-08T13:48:40Z")

</div>

> [@Nick1](#):
>
> Which algorithm is the most appropriate for each case (single node, multi-node, multi-core, etc)?

This completely depends on whether you have big ODEs or small ODEs, whether they have structure to exploit, and how many you need to solve. But generally EnsembleGPUKernel is going to be the fastest for a large number of trajectories if you have access to GPUs and the right methods are implemented in that form (it’s a small subset). Otherwise for big equations you’ll want to look at:

[https://docs.sciml.ai/Overview/stable/showcase/gpu\_spde/](https://docs.sciml.ai/Overview/stable/showcase/gpu_spde/)

---

<div class="post-metadata">

**Author:** ![Nick1](https://avatars.discourse-cdn.com/v4/letter/n/47e85d/32.png) [@Nick1](https://discourse.julialang.org/u/Nick1)\
**Post date:** [March 14, 2023, 4:53pm UTC](https://discourse.julialang.org/t/solving-odes-in-parallel-on-a-supercomputer/95637/6 "2023-03-14T16:53:46Z")

</div>

The system is around 500 ODEs and I am trying to speed up the solving of the system and the GSA as well (with an eFAST algorithm).

---

<div class="post-metadata">

**Author:** ![Nick1](https://avatars.discourse-cdn.com/v4/letter/n/47e85d/32.png) [@Nick1](https://discourse.julialang.org/u/Nick1)\
**Post date:** [March 14, 2023, 4:54pm UTC](https://discourse.julialang.org/t/solving-odes-in-parallel-on-a-supercomputer/95637/7 "2023-03-14T16:54:18Z")

</div>

~500 ODEs and I am also performing GSA using eFAST.

---

<div class="post-metadata">

**Author:** ![Nick1](https://avatars.discourse-cdn.com/v4/letter/n/47e85d/32.png) [@Nick1](https://discourse.julialang.org/u/Nick1)\
**Post date:** [March 14, 2023, 4:55pm UTC](https://discourse.julialang.org/t/solving-odes-in-parallel-on-a-supercomputer/95637/8 "2023-03-14T16:55:00Z")

</div>

Does the same apply to GSA as well?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 18, 2023, 4:01am UTC](https://discourse.julialang.org/t/solving-odes-in-parallel-on-a-supercomputer/95637/9 "2023-03-18T04:01:01Z")

</div>

> [@Nick1](#):
>
> ~500 ODEs and I am also performing GSA using eFAST.

That probably wants EnsembleGPUKernel.