# Some Questions about Parallelization

**URL:** <https://discourse.julialang.org/t/some-questions-about-parallelization/9492>\
**Category:** New to Julia\
**Tags:** parallel\
**Created:** [March 4, 2018, 10:19pm UTC](https://discourse.julialang.org/t/some-questions-about-parallelization/9492 "2018-03-04T22:19:16Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Juser](https://avatars.discourse-cdn.com/v4/letter/j/34f0e0/32.png) [@Juser](https://discourse.julialang.org/u/Juser)\
**Post date:** [March 4, 2018, 10:19pm UTC](https://discourse.julialang.org/t/some-questions-about-parallelization/9492/1 "2018-03-04T22:19:16Z")

</div>

I have a couple questions about parallelism in Julia that I am hoping someone might know the answers to:

(1) Is there an easy way to pass workers a copy of a variable? For example, if on the master process I have created X and would like to create Y=sum(X) on each of the workers, then each of the workers need to see X.

(2) Is there any way to share a vector of shared vectors? I have a number of cases where I have Vector{SharedArray} or even Vector{Vector{SharedArray}} being a convenient way to structure data. However using such storage hurts the parallel performance because of the overhead of passing The Vector or VectorVector part of the above to the worker processes. My current thought is to declare each of these pointing vectors as an @everywhere const so that each processor has a copy, but I’m not sure that this is the cleanest or best way to do this.

Any suggestions or feedback would be appreciated. Thank you!

---

<div class="post-metadata">

**Author:** ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Post date:** [March 5, 2018, 6:49am UTC](https://discourse.julialang.org/t/some-questions-about-parallelization/9492/2 "2018-03-05T06:49:44Z")

</div>

I use  
[https://github.com/ChrisRackauckas/ParallelDataTransfer.jl](https://github.com/ChrisRackauckas/ParallelDataTransfer.jl)  
for the first one.

I do not know the answer for the second.

---

<div class="post-metadata">

**Author:** ![Juser](https://avatars.discourse-cdn.com/v4/letter/j/34f0e0/32.png) [@Juser](https://discourse.julialang.org/u/Juser)\
**Post date:** [March 5, 2018, 4:13pm UTC](https://discourse.julialang.org/t/some-questions-about-parallelization/9492/3 "2018-03-05T16:13:56Z")

</div>

Thanks! This looks like exactly what I need for (1).

---

<div class="post-metadata">

**Author:** ![tk3369](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tk3369/32/2824_2.png) [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Post date:** [March 6, 2018, 6:45am UTC](https://discourse.julialang.org/t/some-questions-about-parallelization/9492/4 "2018-03-06T06:45:33Z")

</div>

I have an interest to share a struct of SharedArray’s as well but from my previous tests it seems to be non-performant to pass that around as you indicated above.

I think the workaround is to create these SharedArray’s individually in the global scope… and if you have many of them then you would come up with a naming convention that distinguish them from each other.

---

<div class="post-metadata">

**Author:** ![Juser](https://avatars.discourse-cdn.com/v4/letter/j/34f0e0/32.png) [@Juser](https://discourse.julialang.org/u/Juser)\
**Post date:** [March 13, 2018, 10:18pm UTC](https://discourse.julialang.org/t/some-questions-about-parallelization/9492/5 "2018-03-13T22:18:00Z")

</div>

I have done some tests, and you’re right. Passing SharedArrays around in vectors or structs leads to extremely poor performance.

Are you suggesting using something that is like a macro to reference the different SharedArrays? The nice thing about vectors of SharedArrays is that I can reference them by their index in my code.

---

<div class="post-metadata">

**Author:** ![tk3369](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tk3369/32/2824_2.png) [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Post date:** [March 14, 2018, 4:56am UTC](https://discourse.julialang.org/t/some-questions-about-parallelization/9492/6 "2018-03-14T04:56:43Z")

</div>

> [@Juser](#):
>
> Passing SharedArrays around in vectors or structs leads to extremely poor performance

For some reasons, I cannot replicate this problem anymore… Do you have any minimum working example (MWE) to share?

---

<div class="post-metadata">

**Author:** ![tk3369](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tk3369/32/2824_2.png) [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Post date:** [March 18, 2018, 4:27am UTC](https://discourse.julialang.org/t/some-questions-about-parallelization/9492/7 "2018-03-18T04:27:05Z")

</div>

> [@tk3369](#):
>
> For some reasons, I cannot replicate this problem anymore

I thought the performance was bad when passing SharedArray’s in another data structure but [my latest test results](https://github.com/tk3369/julia-notebooks/blob/master/Performance%20of%20Different%20SharedArray%20Passing%20Styles.ipynb) do not show any material difference.

---

<div class="post-metadata">

**Author:** ![Juser](https://avatars.discourse-cdn.com/v4/letter/j/34f0e0/32.png) [@Juser](https://discourse.julialang.org/u/Juser)\
**Post date:** [March 18, 2018, 2:51pm UTC](https://discourse.julialang.org/t/some-questions-about-parallelization/9492/8 "2018-03-18T14:51:45Z")

</div>

@tk3369, thanks you for following up on this. I have run similar tests, except passing large numbers of shared arrays one-by-one and without the use of $, and the performance was generally poor. I will not have access to the machine on which I ran the test for a few days, but I will look again and try to understand what is driving the difference in performance. I will try to report back later this week.

---

<div class="post-metadata">

**Author:** ![tk3369](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tk3369/32/2824_2.png) [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Post date:** [March 18, 2018, 5:54pm UTC](https://discourse.julialang.org/t/some-questions-about-parallelization/9492/9 "2018-03-18T17:54:08Z")

</div>

You’re welcome.

> [@Juser](#):
>
> I have run similar tests, except passing large numbers of shared arrays one-by-one and without the use of $, and the performance was generally poor.

You must use `$` to avoid accessing variable in the global scope (which is slow). More details at the [BenchmarkTools README](https://github.com/JuliaCI/BenchmarkTools.jl/blob/master/README.md) file.