# How to efficiently parallelize Julia's code using the same structure across multiple cores

**URL:** <https://discourse.julialang.org/t/how-to-efficiently-parallelize-julias-code-using-the-same-structure-across-multiple-cores/97055>\
**Category:** New to Julia\
**Tags:** question, parallel\
**Created:** [April 4, 2023, 10:09am UTC](https://discourse.julialang.org/t/how-to-efficiently-parallelize-julias-code-using-the-same-structure-across-multiple-cores/97055 "2023-04-04T10:09:12Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![sylowii](https://avatars.discourse-cdn.com/v4/letter/s/50afbb/32.png) [@sylowii](https://discourse.julialang.org/u/sylowii)\
**Post date:** [April 4, 2023, 10:09am UTC](https://discourse.julialang.org/t/how-to-efficiently-parallelize-julias-code-using-the-same-structure-across-multiple-cores/97055/1 "2023-04-04T10:09:12Z")

</div>

Hi all,

I am trying to efficiently parallelize a code that relies on a big data structure (let’s say matrices of size 10^5 x 10^3 ). A small example is provided below. I would like to avoid creating/copying the same data structures on each core, instead, it would be convenient to use the same structure in parallel.  
The iterative code is the following:

```julia
# create remote data structure
struct complex_mvts
    raw_data::Matrix{Float64}
    num_ROI::Int
    T::Int
end

# create remote function
function create_data_structure(data::Matrix{Float64})
    N,T=size(data)
    return(complex_mvts(data,N,T))
end

# define launch function
function launch_one_t(TS::complex_mvts,t::Int64)
    a=sum(TS.raw_data[:,t])
    println(a)
end

# main function
function main()

    # create local data structure
    data = rand(100000,1000)
    TS=create_data_structure(data)

    # launch function
    for t in 1:1000
        launch_one_t(TS,t)
    end
end

# call main function
main()

```

I have tried to use `Distributed` but it doesn’t seem to provide any kind of speedup (actually is far slower). Do you have any suggestions?

```julia
# import Distributed library
using Distributed

# add worker processes
addprocs(4)

# create remote data structure
@everywhere struct complex_mvts
    raw_data::Matrix{Float64}
    num_ROI::Int
    T::Int
end

# create remote function
@everywhere function create_data_structure(data::Matrix{Float64})
    N,T=size(data)
    return(complex_mvts(data,N,T))
end

# define launch function
@everywhere function launch_one_t(TS::complex_mvts,t::Int64)
    s=sum(TS.raw_data[:,t])
    print(s, " ")
end

# main function
function main()

    # create local data structure
    data = rand(100000,1000)
    TS=create_data_structure(data)
    # launch function in parallel
    @sync @distributed for t in 1:100
        launch_one_t(TS,t)
    end
end

# call main function
main()

```

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [April 4, 2023, 11:06am UTC](https://discourse.julialang.org/t/how-to-efficiently-parallelize-julias-code-using-the-same-structure-across-multiple-cores/97055/2 "2023-04-04T11:06:46Z")

</div>

The main difference between flavors of parallel computing is that multiprocessing (with `Distributed`) relies on distributed memory, whereas multithreading (with `Base.Threads`) relies on shared memory: see the [documentation page](https://docs.julialang.org/en/v1/manual/parallel-computing/). So in many cases, the data transfer alone will make `Distributed` much slower. Have you tried plain multithreading?

---

<div class="post-metadata">

**Author:** ![sylowii](https://avatars.discourse-cdn.com/v4/letter/s/50afbb/32.png) [@sylowii](https://discourse.julialang.org/u/sylowii)\
**Post date:** [April 4, 2023, 11:57am UTC](https://discourse.julialang.org/t/how-to-efficiently-parallelize-julias-code-using-the-same-structure-across-multiple-cores/97055/3 "2023-04-04T11:57:52Z")

</div>

Thanks a lot for the info! I’ve had a look at the documentation and even when using Threads, it seems that there is no significant speed-up even using plain multithreading. I tried wrapping the for loop (with `Base.Threads`) as follows:

```julia
@threads for t in 1:1000
        launch_one_t(t)
    end

```

I come from Python and in the past, I used global variables there for the same problem, but I know that in Julia global variables should be avoided when possible.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [April 4, 2023, 12:38pm UTC](https://discourse.julialang.org/t/how-to-efficiently-parallelize-julias-code-using-the-same-structure-across-multiple-cores/97055/4 "2023-04-04T12:38:30Z")

</div>

What’s your `Threads.nthreads()`? Also your slice `TS.raw_data[:,t]` is allocating a temporary copy, that could be slowing things down. Pretty sure the `@view(TS.raw_data[:,t])` is contiguous so it shouldn’t slow the `sum` down, should check though.

---

<div class="post-metadata">

**Author:** ![sylowii](https://avatars.discourse-cdn.com/v4/letter/s/50afbb/32.png) [@sylowii](https://discourse.julialang.org/u/sylowii)\
**Post date:** [April 4, 2023, 1:31pm UTC](https://discourse.julialang.org/t/how-to-efficiently-parallelize-julias-code-using-the-same-structure-across-multiple-cores/97055/5 "2023-04-04T13:31:41Z")

</div>

I was using Jupiter for launching the code, so I had 1 thread available. I did run some tests using 12 cores (both Jupiter and shell using the command `julia --threads 12` ) and there is a slight speed-up. It also helped the `@view(TS.raw_data[:,t])`. Here is the code that I am running for reference:

```julia
using Base.Threads
using BenchmarkTools

# create remote data structure
struct complex_mvts
    raw_data::Matrix{Float64}
    num_ROI::Int
    T::Int
end

# create remote function
function create_data_structure(data::Matrix{Float64})
    N,T=size(data)
    return(complex_mvts(data,N,T))
end

# define launch function
function launch_one_t(TS::complex_mvts,t::Int64)
    s=sum(@view(TS.raw_data[:,t]))
    # print(s, " ")
end

# Iterative function
function iterative_func()

    # create local data structure
    data = rand(10000,2500)
    TS=create_data_structure(data)

    # launch function in parallel
    for t in 1:2500
        launch_one_t(TS,t)
    end
end

# Parallel function
function func_parallel()

    # create local data structure
    data = rand(10000,2500)
    TS=create_data_structure(data)

    # launch function in parallel
    @threads for t in 1:2500
        launch_one_t(TS,t)
    end
end

```

With the iterative, using `@benchmark iterative_func()` I get this:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/9/e/9e023b3d46cc6fb1be0ecccca24d3ba7785d943b.png)

While for the parallel, using `@benchmark func_parallel()` I get this:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/e/2/e2f051ed5d07f2a86933ae38927ab026f79f8fb3.png)

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [April 4, 2023, 11:13pm UTC](https://discourse.julialang.org/t/how-to-efficiently-parallelize-julias-code-using-the-same-structure-across-multiple-cores/97055/6 "2023-04-04T23:13:25Z")

</div>

A problem with the benchmark is that it’s including the construction of the input data, so the same allocation and GC overhead is being added to the timing. Ideally you isolate the different parts (the for loop) into their own functions and benchmark those.

It’s also limiting the timing improvements. `rand(10000, 2500)` takes ~7x longer than 2500 (serial) sums of 10000 values (on v1.8.5), in other words the serial loop was ~1/8 of the runtime. By Amdahl’s law, a full 12x (actually less due to multithreading overhead) speedup of the loop would result in 1/((1-1/8)+1/8/12)=1.1294x speedup overall, approaching 1/(1-1/8)=1.1429x at infinite cores.

---

<div class="post-metadata">

**Author:** ![sylowii](https://avatars.discourse-cdn.com/v4/letter/s/50afbb/32.png) [@sylowii](https://discourse.julialang.org/u/sylowii)\
**Post date:** [April 5, 2023, 7:05am UTC](https://discourse.julialang.org/t/how-to-efficiently-parallelize-julias-code-using-the-same-structure-across-multiple-cores/97055/7 "2023-04-05T07:05:25Z")

</div>

Thanks a lot for the explanation. I benchmarked the functions when isolating the different parts of the loop. It is now much more evident that there is a speed-up. Here are the results:

```julia
@btime iterative_func(data)
  9.473 ms (0 allocations: 0 bytes)

@btime func_parallel(data)
6.161 ms (73 allocations: 6.52 KiB)

```