# Writing parallel for loops

**URL:** <https://discourse.julialang.org/t/writing-parallel-for-loops/7025>\
**Category:** New to Julia\
**Created:** [November 12, 2017, 1:09pm UTC](https://discourse.julialang.org/t/writing-parallel-for-loops/7025 "2017-11-12T13:09:28Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![villageidiot](https://avatars.discourse-cdn.com/v4/letter/v/e68b1a/32.png) [@villageidiot](https://discourse.julialang.org/u/villageidiot)\
**Post date:** [November 12, 2017, 1:09pm UTC](https://discourse.julialang.org/t/writing-parallel-for-loops/7025/1 "2017-11-12T13:09:28Z")

</div>

I’ve got a program where I want to parallelize some for loops. If I add the parallelize decorator will it automatically detect my number of cores and use them? Or do I have to declare it somewhere beforehand? The examples in the docs I saw are geared towards running stuff from the interpreter.

If I define a `SharedArray` for use in parallel processing (as mentioned in the docs) is there any downside downstream? I assume its type is still `SharedArray` after the parallel processing. Basically can I use a `SharedArray` just like a normal Array in linear processing?

Finally, is there a nice way to switch between linear and parallel processing? Ideally it’d be something like  
`@parallel(len_data > 1000)`, might there be something like that which would allow one to avoid having to write code for linear and parallel use?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [November 12, 2017, 2:23pm UTC](https://discourse.julialang.org/t/writing-parallel-for-loops/7025/2 "2017-11-12T14:23:15Z")

</div>

> [@villageidiot](#):
>
> If I add the parallelize decorator will it automatically detect my number of cores and use them?

No, check out the documentation parts on `addprocs` or opening Julia with `julia -p n`

> [@villageidiot](#):
>
> If I define a SharedArray for use in parallel processing (as mentioned in the docs) is there any downside downstream?

Yes. `SharedArray`s share the data, so they are not as fast in many cases as doing a strictly parallel operation without it. Many times it can be faster to just write an algorithm that splits off the data to processes exactly the way you need it, and do the parallel reduction on independent pieces. YMMV.

> [@villageidiot](#):
>
> Finally, is there a nice way to switch between linear and parallel processing? Ideally it’d be something like
> 
> @parallel(len\_data \> 1000), might there be something like that which would allow one to avoid having to write code for linear and parallel use?

A conditional? Or write a macro that spits this kind of code out. But you have to be careful because `@parallel` isn’t exactly the same as a loop.

---

<div class="post-metadata">

**Author:** ![villageidiot](https://avatars.discourse-cdn.com/v4/letter/v/e68b1a/32.png) [@villageidiot](https://discourse.julialang.org/u/villageidiot)\
**Post date:** [November 19, 2017, 5:31pm UTC](https://discourse.julialang.org/t/writing-parallel-for-loops/7025/4 "2017-11-19T17:31:21Z")

</div>

I’m still confused as to how to declare which parts of a program are accessible or not from the different workers. On a relatively low level in a program of mine I have a for loop I want to parallelize.

```
module module_name
[...]
addprocs(3)
totalsum = @parallel (+) for i in 1:large_number
    tmp_sum = 0
    for j in 1:num
        ... # calls f1 f2
    end
    tmp_sum # not sure how to 'return' the result, the examples have a conveniantly placed calculation at the end
end
rmprocs([2 3 4])
[...]
end

```

As I understand I’d have to put the `@everywhere` decorator infront of `f1` and `f2`. But the program fails far before with the additional workers complaining that `UndefVarError: module_name not defined`, and I have no clue how to fix that.

I feel like I’ve missed something needed for setting up parallel processing. As I understood it other than writing the actual `@parallel` part, one needs to `addprocs` and then add the `@everywhere` decorator to those functions used inside the loop. Is that really it?

I know `pmap` is better suited for what I’m doing here but I wanted to get the simpler option to work first (I’d need to pass several arguments the `pmap` function).

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [November 20, 2017, 2:32pm UTC](https://discourse.julialang.org/t/writing-parallel-for-loops/7025/5 "2017-11-20T14:32:28Z")

</div>

> [@villageidiot](#):
>
> But the program fails far before with the additional workers complaining that UndefVarError: module\_name not defined, and I have no clue how to fix that.

You defined the module in the global scope so like all others in the global namespace, you need to make sure it’s on the other processes… so `@everywhere` it. Packages are exempt from this because `using` knows to check the path directly and so each can import the module itself.

> [@villageidiot](#):
>
> I feel like I’ve missed something needed for setting up parallel processing. As I understood it other than writing the actual @parallel part, one needs to addprocs and then add the @everywhere decorator to those functions used inside the loop. Is that really it?

Yup. If you’ve defined each process, and each process has the required functions, then it will run in parallel just fine.