# Parallel loops and maps
**URL:** https://discourse.julialang.org/t/parallel-loops-and-maps/39309
**Category:** General Usage
**Tags:** question
**Created:** [May 11, 2020, 9:25pm UTC](https://discourse.julialang.org/t/parallel-loops-and-maps/39309 "2020-05-11T21:25:21Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@Jing\_Zhang](https://discourse.julialang.org/u/Jing_Zhang)
#### Post date: [May 11, 2020, 9:25pm UTC](https://discourse.julialang.org/t/parallel-loops-and-maps/39309/1 "2020-05-11T21:25:21Z")
Dear Julia Users,
I am running multiple layers of for loops. Within the for loops, it is a function named do\_one, which will return a 105000×34 DataFrame. I would want to combine these DataFrame to a larger DataFrame. Codes are as below
results\_all = Array{Any}(undef, I, J, K)
for i = 1: I
for j=1:J
for k=1:K
result = do\_one(…)
results\_all[i, j, k] = result
end
end
end
My question: how to realize parallel computing with @distributed or pmap?
Thank you so much!
---
### Author:  [@anon94023334](https://discourse.julialang.org/u/anon94023334)
#### Post date: [May 11, 2020, 10:07pm UTC](https://discourse.julialang.org/t/parallel-loops-and-maps/39309/2 "2020-05-11T22:07:14Z")
what are you passing to `do_one`? Is it `i, j`, or `j, k`,? That is, are you getting `K` dataframes, or are you getting `I` dataframes?
Also, please enclose your code in triple backticks:
```
`code goes here`
```
---
### Author:  [@Jing\_Zhang](https://discourse.julialang.org/u/Jing_Zhang)
#### Post date: [May 11, 2020, 10:37pm UTC](https://discourse.julialang.org/t/parallel-loops-and-maps/39309/4 "2020-05-11T22:37:47Z")
i, j, k are all passed to do\_one through “parameters.jl”. Each i, j, k combination will produce a 105000×34 DataFrame, i.e., result in the code. But results\_all should be I _J_ K array, with each of its element being a a 105000×34 DataFrame.
```julia
results_all = Array{Any}(undef, I, J, K)
@everywhere include("parameters.jl")
for i = 1:I
for j=1:J
for k=1:K
result = do_one(…)
results_all[i, j, k] = result
end
end
end
```
---
### Author:  [@Jing\_Zhang](https://discourse.julialang.org/u/Jing_Zhang)
#### Post date: [May 11, 2020, 10:49pm UTC](https://discourse.julialang.org/t/parallel-loops-and-maps/39309/5 "2020-05-11T22:49:40Z")
I also should point out that different parameters will be passed to do\_one in different i, j, k combinations.
parameters.jl contains information of parameter1, parameter2, parameter3.
So the codes are
```julia
results_all = Array{Any}(undef, I, J, K)
@everywhere include("parameters.jl")
for i = 1:I
for j=1:J
for k=1:K
result = do_one(parameter1[i], parameter2[j], parameter3[k])
results_all[i, j, k] = result
end
end
end
```
---
### Author:  [@anon94023334](https://discourse.julialang.org/u/anon94023334)
#### Post date: [May 11, 2020, 10:50pm UTC](https://discourse.julialang.org/t/parallel-loops-and-maps/39309/6 "2020-05-11T22:50:54Z")
So, I would do something like this. Note that I think you shouldn’t use `Any`. You should use a real type.
```julia
results_all = SharedArray{Int}(I, J, K)
@sync @distributed for i = 1:I
for j = 1:J, k = 1:K
result = do_one(...)
results_all[i, j, k] = result
end
end
```
You might want to move the distributed for to one of the inner loops, but that’s how to parallelize it.
---
### Author:  [@Jing\_Zhang](https://discourse.julialang.org/u/Jing_Zhang)
#### Post date: [May 11, 2020, 11:04pm UTC](https://discourse.julialang.org/t/parallel-loops-and-maps/39309/7 "2020-05-11T23:04:44Z")
Thanks a lot! Since you only put @sync @distributed ONLY for the I iterations, each j and k combination will be conducted on the same core? I.e., there are I parallel jobs.
Can I do the following codes to run I_J_K parallel jobs?
```julia
results_all = SharedArray{Int}(I, J, K)
@sync @distributed for i = 1:I
@sync @distributed for j = 1:J
@sync @distributed k = 1:K
result = do_one(...)
results_all[i, j, k] = result
end
end
end
```
---
### Author:  [@Jing\_Zhang](https://discourse.julialang.org/u/Jing_Zhang)
#### Post date: [May 12, 2020, 12:56am UTC](https://discourse.julialang.org/t/parallel-loops-and-maps/39309/8 "2020-05-12T00:56:36Z")
Another problem is that the `result = do_one(parameter1[i], parameter2[j], parameter3[k])` returns a data frame with elements of mixed types, including Float64, String, and Int64. So I cannot use `results_all = SharedArray{Int}(I, J, K)`, then what type I should use here? I cannot do `results_all = SharedArray{Any}(I, J, K)`.
---
### Author:  [@anon94023334](https://discourse.julialang.org/u/anon94023334)
#### Post date: [May 12, 2020, 1:37am UTC](https://discourse.julialang.org/t/parallel-loops-and-maps/39309/9 "2020-05-12T01:37:19Z")
instead of `Int`, perhaps you can put in the type of the dataframe then? At this point you’re probably just going to have to experiment with possibilities since we’re beyond a MWE.
---
### Author:  [@Jing\_Zhang](https://discourse.julialang.org/u/Jing_Zhang)
#### Post date: [May 12, 2020, 1:47am UTC](https://discourse.julialang.org/t/parallel-loops-and-maps/39309/10 "2020-05-12T01:47:06Z")
Thanks! I think `SharedArray` may not work for an array with mixed types of data, including Float64, Int64 and String.
There are probably other functions work like `SharedArray`.
Or I will need to look at `pmap` instead.