# Run model runs in a parallel for loop in julia

**URL:** https://discourse.julialang.org/t/run-model-runs-in-a-parallel-for-loop-in-julia/6326
**Category:** Julia at Scale
**Created:** [October 8, 2017, 2:45pm UTC](https://discourse.julialang.org/t/run-model-runs-in-a-parallel-for-loop-in-julia/6326 "2017-10-08T14:45:43Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Simon\_Besnard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simon_besnard/32/9730_2.png) [@Simon\_Besnard](https://discourse.julialang.org/u/Simon_Besnard)
#### Post date: [October 8, 2017, 2:45pm UTC](https://discourse.julialang.org/t/run-model-runs-in-a-parallel-for-loop-in-julia/6326/1 "2017-10-08T14:45:43Z")

</div>

I am trying to parallelize the code below. The idea is to run several model runs in parallel and push their output in to a single dataframe (i.e. dout). The code I have created works with no parrallelisation so far. However, I would be interested in making it faster as at the end I would have more than 1000 model runs. Any idea how I could parralelize this piece of code.

```julia
dout=DataFrame[]
for i= 1:l10
model=train(y,x);
ynew= predict(model, x[i)
Prediction_df = DataFrame(Prediction=mapreduce(vec, vcat, ynew))
push!(dout,DataFrame(Prediction_df)
end

```

---

<div class="post-metadata">

### Author: ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)
#### Post date: [October 8, 2017, 6:44pm UTC](https://discourse.julialang.org/t/run-model-runs-in-a-parallel-for-loop-in-julia/6326/2 "2017-10-08T18:44:07Z")

</div>

I don’t want to sound too harsh, but you _really_ need to improve your question if you want a decent answer. I will give you three reasons why I say this:

1. The provided example does not give any indication of what’s going on, other than there being a loop where you train, predict, and map-reduce each iteration. It’s hardly enough information for me to say anything other than: “Run it in a @threads for loop”. This is especially true when trying to convert serial code to parallel, as there are a huge range of factors to consider.
2. You need to give a more elaborate answer which indicates what `train` and `predict` do and where they come from, so we can inspect those functions and their containing modules/libraries. Or even better, boil your example down into proper MWE (Minimum Working Example) that anyone seeing your answer can run on their local machine.
3. The example itself does not even run, even if all functions being used were already known and defined. I see errors on lines 2, 4, and 6 which would make your code non-runnable. While this may sound like me nitpicking, proper syntax that “just works” for people trying to assist you goes a very long way.

I would recommend you either edit your original question, or create a new one, with the above 3 points addressed. Only then will people be willing and able to assist you.

---

<div class="post-metadata">

### Author: ![jonathanBieler](https://avatars.discourse-cdn.com/v4/letter/j/82dd89/32.png) [@jonathanBieler](https://discourse.julialang.org/u/jonathanBieler)
#### Post date: [October 8, 2017, 6:56pm UTC](https://discourse.julialang.org/t/run-model-runs-in-a-parallel-for-loop-in-julia/6326/3 "2017-10-08T18:56:36Z")

</div>

I think you can use `pmap` here, you need to write a function that takes a `x` and return your dataframe and then map x on it with `pmap`, here’s a minimal example:

```julia
addprocs(2)

x = rand(10)
@everywhere begin
    using DataFrames
    model(x,other_para) = DataFrame(val=x+other_para,id=myid())
end

pmap(x->model(x,1),x)

```

[https://docs.julialang.org/en/latest/manual/parallel-computing#Parallel-Map-and-Loops-1](https://docs.julialang.org/en/latest/manual/parallel-computing#Parallel-Map-and-Loops-1)

Threads.@threads might also just work, something like:

```julia
Threads.@threads for i=1:10
    out[i] = somefun(x[i])
end

```

But you need to start Julia with several threads, and it can make Julia crash if you read files or things like that (look up the docs).
