# Help me process this data in parallel

**URL:** <https://discourse.julialang.org/t/help-me-process-this-data-in-parallel/95562>\
**Category:** General Usage\
**Tags:** parallel\
**Created:** [March 5, 2023, 1:14am UTC](https://discourse.julialang.org/t/help-me-process-this-data-in-parallel/95562 "2023-03-05T01:14:21Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![marianoarnaiz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marianoarnaiz/32/19377_2.png) [@marianoarnaiz](https://discourse.julialang.org/u/marianoarnaiz)\
**Post date:** [March 5, 2023, 1:14am UTC](https://discourse.julialang.org/t/help-me-process-this-data-in-parallel/95562/1 "2023-03-05T01:14:21Z")

</div>

Hi everyone.  
I have a simple function that turns each row of a matrix into a seismic trace (Seis.jl format) and filter it.

Can you guys help me turn this into a parallel function? Like several cores filtering one trace at the same time? I have 8 cores i guess I can run this is several of them to gain some speed!

Thanks a lot.

> function bpdas(strainrate, time, offset, f1, f2; poles=2)  
> # Begin  
> bpstrainrate=copy(strainrate);  
> #set up some trace information  
> b=0.0;  
> delta=time[2]-time[1];  
> # Bandpass Filtering Loop  
> for i =1:size(offset,1)  
> dastrace=Trace(b, delta, strainrate[i,:]) #write the data to trace format  
> bandpass!(dastrace, f1, f2; poles) #filter in place  
> bpstrainrate[i,:]=trace(dastrace); #safe to global variable  
> end  
> return bpstrainrate  
> end

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [March 5, 2023, 2:04am UTC](https://discourse.julialang.org/t/help-me-process-this-data-in-parallel/95562/2 "2023-03-05T02:04:40Z")

</div>

```julia
function bpdas(strainrate, time, offset, f1, f2; poles=2)
# Begin
    bpstrainrate=copy(strainrate);
#Set up some trace information
    b=0.0;
    delta=time[2]-time[1];
# Bandpass Filtering Loop
    Base.Threads.@threads for i =1:size(offset,1)
        dastrace=Trace(b, delta, strainrate[i,:]) #write the data to trace format
        bandpass!(dastrace, f1, f2; poles) #filter in place
        bpstrainrate[i,:]=trace(dastrace); #safe to global variable
    end
    return bpstrainrate
end

```

Provided I understand the data.

---

<div class="post-metadata">

**Author:** ![marianoarnaiz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marianoarnaiz/32/19377_2.png) [@marianoarnaiz](https://discourse.julialang.org/u/marianoarnaiz)\
**Post date:** [March 5, 2023, 7:56am UTC](https://discourse.julialang.org/t/help-me-process-this-data-in-parallel/95562/3 "2023-03-05T07:56:15Z")

</div>

I will give it a go :).  
Is it necesary to start julia like this for it to work

> julia -p 6

---

<div class="post-metadata">

**Author:** ![marianoarnaiz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marianoarnaiz/32/19377_2.png) [@marianoarnaiz](https://discourse.julialang.org/u/marianoarnaiz)\
**Post date:** [March 5, 2023, 8:12am UTC](https://discourse.julialang.org/t/help-me-process-this-data-in-parallel/95562/4 "2023-03-05T08:12:12Z")

</div>

> [@PetrKryslUCSD](#):
>
> `Base.Threads.@threads`

Just it case, all the rows strainrate[i,:] are independent from each other so every loop repetition is also independet. If that helps hehehe

---

<div class="post-metadata">

**Author:** ![jmair](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmair/32/35117_2.png) [@jmair](https://discourse.julialang.org/u/jmair)\
**Post date:** [March 5, 2023, 10:17am UTC](https://discourse.julialang.org/t/help-me-process-this-data-in-parallel/95562/5 "2023-03-05T10:17:38Z")

</div>

This is multiprocessing _not_ multithreading (which is what `Threads.@threads` uses). You should only really use one or the other on a single machine.

In your case, you would need to start with more threads, e.g:

```sh
julia --threads=auto

```

Or to start with `4` threads

```sh
julia --threads=4

```

This can also be done with an environment variable, or there’s even a setting in VS code for it too.

---

<div class="post-metadata">

**Author:** ![marianoarnaiz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marianoarnaiz/32/19377_2.png) [@marianoarnaiz](https://discourse.julialang.org/u/marianoarnaiz)\
**Post date:** [March 6, 2023, 9:21am UTC](https://discourse.julialang.org/t/help-me-process-this-data-in-parallel/95562/6 "2023-03-06T09:21:16Z")

</div>

> [@jmair](#):
>
> `threads`

If I want to process each i in a different hardware (processor, thread, worker, i don’t know)… is the macro doing this?

---

<div class="post-metadata">

**Author:** ![jmair](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmair/32/35117_2.png) [@jmair](https://discourse.julialang.org/u/jmair)\
**Post date:** [March 6, 2023, 9:27am UTC](https://discourse.julialang.org/t/help-me-process-this-data-in-parallel/95562/7 "2023-03-06T09:27:01Z")

</div>

Use `Threads.@threads` on the front of a `for` loop. If you want to use functions like `map` & `reduce` there is a package called `ThreadsX.jl` which provides parallel implementations of these base library functions as well.

E.g.

```julia
function double!(array)
    Threads.@threads for i in eachindex(array)
        array[i] = array[i] * 2
    end
    return nothing
end

```