# Comparison of languages for parallel computing tasks

**URL:** <https://discourse.julialang.org/t/comparison-of-languages-for-parallel-computing-tasks/46806>\
**Category:** Performance\
**Created:** [September 17, 2020, 10:13pm UTC](https://discourse.julialang.org/t/comparison-of-languages-for-parallel-computing-tasks/46806 "2020-09-17T22:13:49Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [September 17, 2020, 10:13pm UTC](https://discourse.julialang.org/t/comparison-of-languages-for-parallel-computing-tasks/46806/1 "2020-09-17T22:13:49Z")

</div>

The paper  
[https://www.sciencedirect.com/science/article/abs/pii/S2210650220303734](https://www.sciencedirect.com/science/article/abs/pii/S2210650220303734)  
[https://lilloa.univ-lille.fr/handle/20.500.12210/29528](https://lilloa.univ-lille.fr/handle/20.500.12210/29528)  
compares Chapel, Python, and Julia. For reasons that are at this point not crisp to me  
Julia has not performed well.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [September 17, 2020, 10:25pm UTC](https://discourse.julialang.org/t/comparison-of-languages-for-parallel-computing-tasks/46806/2 "2020-09-17T22:25:22Z")

</div>

~~They don’t provide any of the code that they used for the comparison, making it pretty hard to evaluate this article independently.~~ [Here is the code on Zenodo](https://zenodo.org/record/3765373), and [here is their github repo](https://github.com/jangmys/BenchParallelMeta).

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [September 17, 2020, 10:41pm UTC](https://discourse.julialang.org/t/comparison-of-languages-for-parallel-computing-tasks/46806/3 "2020-09-17T22:41:14Z")

</div>

There is a zenodo link to the complete codes.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [September 17, 2020, 11:03pm UTC](https://discourse.julialang.org/t/comparison-of-languages-for-parallel-computing-tasks/46806/4 "2020-09-17T23:03:19Z")

</div>

[https://github.com/jangmys/BenchParallelMeta/blob/d356be83e1f1e7c27052456e8300f5982db1b6b6/Q3AP-ILS/Julia/ils\_q3ap\_par.jl#L129](https://github.com/jangmys/BenchParallelMeta/blob/d356be83e1f1e7c27052456e8300f5982db1b6b6/Q3AP-ILS/Julia/ils_q3ap_par.jl#L129)

looks like there’s no warm-up

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [September 17, 2020, 11:05pm UTC](https://discourse.julialang.org/t/comparison-of-languages-for-parallel-computing-tasks/46806/5 "2020-09-17T23:05:31Z")

</div>

Q3AP-ILS unmodified:

```julia
% julia -t 4 ./ils_q3ap_par.jl nug12 100
Time (Init instance): 1.764211577
Time (First LS/Compilation): 0.230459977
======== ITERATED LOCAL SEARCH ======== 100
        Best solution

Solution([5, 4, 7, 6, 10, 11, 12, 9, 2, 1, 3, 8], [11, 7, 6, 2, 12, 4, 1, 9, 8, 5, 10, 3], 658)

        TotalTime: 3.390096732
        ILSTime: 1.395425178
        Nhood evals: 533
        NhoodPerSec: 381.96243582470316

```

with a bunch of `const` to avoid using global non-constant variables:

```julia
% julia -t 4 ./ils_q3ap_par.jl nug12 100
Time (Init instance): 1.796165105
Time (First LS/Compilation): 0.202069824
WARNING: redefinition of constant sol. This may fail, cause incorrect answers, or produce other errors.
======== ITERATED LOCAL SEARCH ======== 100
WARNING: redefinition of constant sol. This may fail, cause incorrect answers, or produce other errors.
        Best solution

Solution([6, 10, 9, 3, 7, 2, 12, 1, 8, 11, 4, 5], [6, 7, 4, 5, 2, 9, 11, 10, 8, 3, 12, 1], 778)

        TotalTime: 3.181146955
        ILSTime: 1.182912026
        Nhood evals: 528
        NhoodPerSec: 446.3561012101842

```

Running all the `@elapsed ...` twice to not measure compilation:

```julia
% julia -t 4 ./ils_q3ap_par.jl nug12 100
Time (Init instance): 0.065759911
Time (First LS/Compilation): 0.001595556
WARNING: redefinition of constant sol. This may fail, cause incorrect answers, or produce other errors.
======== ITERATED LOCAL SEARCH ======== 100
WARNING: redefinition of constant sol. This may fail, cause incorrect answers, or produce other errors.
======== ITERATED LOCAL SEARCH ======== 100
WARNING: redefinition of constant sol. This may fail, cause incorrect answers, or produce other errors.
        Best solution

Solution([4, 9, 2, 8, 5, 6, 7, 3, 1, 10, 11, 12], [8, 4, 1, 12, 5, 11, 10, 3, 6, 7, 2, 9], 752)

        TotalTime: 1.141571685
        ILSTime: 1.074216218
        Nhood evals: 544
        NhoodPerSec: 506.41573910774827

```

`NhoodPerSec` increases by more than 32% with two trivial changes

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [September 18, 2020, 12:33am UTC](https://discourse.julialang.org/t/comparison-of-languages-for-parallel-computing-tasks/46806/6 "2020-09-18T00:33:11Z")

</div>

Are/have any of the authors been active here? In particular, it would be great to get some clarification on these statements:

> Consequently, much of the information one can find in online forums and documentation is no longer valid

> There is a large amount of documentation available, but it sometimes feels opaque—for instance we were unable to find information on the thread layer used for the multi-threading package.

Also, using globals is mentioned as a performance trap so often that not accounting for them and then claiming poor performance seems suspect. If nothing else, it makes this section sound like a cop-out:

> In our case, both programmers have strong prior experience with C and parallel computing and little to intermediate prior knowledge of Python, Julia and Chapel. As detailed in Section 4, we have followed a protocol that aims at making the comparison fair. However, we cannot completely exclude that some parts of the code could be written more efficiently or concisely.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [September 18, 2020, 12:37am UTC](https://discourse.julialang.org/t/comparison-of-languages-for-parallel-computing-tasks/46806/7 "2020-09-18T00:37:26Z")

</div>

> [@ToucheSir](#):
>
> > [@](#):
> >
> > Consequently, much of the information one can find in online forums and documentation is no longer valid

To put into context, the paper was submitted on 15 October 2019, which is almost one year ago.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [September 18, 2020, 12:45am UTC](https://discourse.julialang.org/t/comparison-of-languages-for-parallel-computing-tasks/46806/8 "2020-09-18T00:45:40Z")

</div>

I can’t help but feel if they know to use `Numpy` and even `Numba` for Python (at which point it’s no longer benchmarking python anymore), at least they should know to time things properly? (with worm up etc.), and read the performance tips Manual page once?

For example, I won’t expect them to know to use `@SVector`, as a comparison. But come on, if the paper is about comparing\* performance, at least get that little bit right.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [September 18, 2020, 1:14am UTC](https://discourse.julialang.org/t/comparison-of-languages-for-parallel-computing-tasks/46806/9 "2020-09-18T01:14:38Z")

</div>

Concerning warmup: If the Julia code is cold-started, then it would be fair  
to include in the time comparison also the compile/link times  
for Chapel and C/OpenMP, wouldn’t it?

---

<div class="post-metadata">

**Author:** ![ImreSamu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/imresamu/32/20677_2.png) [@ImreSamu](https://discourse.julialang.org/u/ImreSamu)\
**Post date:** [September 18, 2020, 1:32am UTC](https://discourse.julialang.org/t/comparison-of-languages-for-parallel-computing-tasks/46806/10 "2020-09-18T01:32:53Z")

</div>

> [@PetrKryslUCSD](#):
>
> For reasons that are at this point not crisp to me  
> Julia has not performed well.

As in the paper :

- **Julia 1.2**
- page36: _"Numba’s and Julia’s (experimental) multi-threading support is not mature_  
_enough to compete with OpenMP or Chapel in terms of scalability."_

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [September 18, 2020, 2:39am UTC](https://discourse.julialang.org/t/comparison-of-languages-for-parallel-computing-tasks/46806/11 "2020-09-18T02:39:15Z")

</div>

True for Julia at the time. It might be interesting to re-run with 1.6.

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [September 21, 2020, 9:11am UTC](https://discourse.julialang.org/t/comparison-of-languages-for-parallel-computing-tasks/46806/12 "2020-09-21T09:11:40Z")

</div>

I am going to show some prejudice here. Chapel has been around for a long time. It is associated with Cray of course. I do not know if it will ever break out beyond that.  
I can be very wrong - I am not plugged into that community.