# Parallelization of for loop

**URL:** <https://discourse.julialang.org/t/parallelization-of-for-loop/70557>\
**Category:** General Usage\
**Tags:** parallel, multithreading, tullio, loopvectorization\
**Created:** [October 28, 2021, 2:36pm UTC](https://discourse.julialang.org/t/parallelization-of-for-loop/70557 "2021-10-28T14:36:48Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Germinator](https://avatars.discourse-cdn.com/v4/letter/g/3be4f8/32.png) [@Germinator](https://discourse.julialang.org/u/Germinator)\
**Post date:** [October 28, 2021, 2:36pm UTC](https://discourse.julialang.org/t/parallelization-of-for-loop/70557/1 "2021-10-28T14:36:48Z")

</div>

Dear Julia users,

I am trying to parallelize the following loop, what do you think is the best approach?

```julia
         for n = 1:N
             kron!(temp,A[n,:],B[n,:])
             C.+= temp.*temp'
             Cx .+= temp.*x[n]
         end

```

I tried using FLoops, but the outputs differ, and the code is way slower:

```julia
        for n = 1:N
            kron!(temp,A[n,:],B[n,:])
            @reduce(C= zeros(M,M) + temp*temp', Cx = zeros(M) + temp*x[n])
        end   

```

I am stuck and would appreciate any kind of help!

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [October 29, 2021, 6:23am UTC](https://discourse.julialang.org/t/parallelization-of-for-loop/70557/2 "2021-10-29T06:23:24Z")

</div>

> [@Germinator](#):
>
> `A[n,:],B[n,:]`

Generally, before you parallelize a code, you want to make sure that the serial performance is decent. In particular, you want to avoid unnecessary allocations. Matrix slices allocate in Julia by default, so you want to put `@views` in front of the `kron!`.

Also you might want to transpose `A` and `B`, since operating on a sliced column should be faster than operating on a sliced row. Like Fortran/MATLAB and in contrast to C/Python, Julia is column major order.

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [October 29, 2021, 12:00pm UTC](https://discourse.julialang.org/t/parallelization-of-for-loop/70557/3 "2021-10-29T12:00:02Z")

</div>

The difficulty with parallelising the loop over `n` is that you need somewhere to accumulate the results of each thread, to combine at the end.

I think it’ll work better to do the other loops in parallel. And to make the one that is summing the innermost. In which case this layout of the arrays probably isn’t a bad thing. But you will want to avoid making all these slices, and deal directly with the elements.

Here’s one way:

```julia
N = 5; M = 2;
A = rand(N,M); B = rand(N,M); x = rand(N); 
C = zeros(M^2, M^2); Cx = zeros(M^2); temp = rand(M^2);

function orig!(C,Cx,A,B,temp)
    for n in 1:size(A,1)
        @views kron!(temp,A[n,:],B[n,:])
        C.+= temp.*temp'
        Cx .+= temp.*x[n]
    end
end
orig!(C,Cx,A,B,temp)

using Tullio # should parallelise, almost 4x quicker for me
# using LoopVectoriztion # another 3x or so speed up
@tullio C4[j,i,l,k] := A[n,i] * B[n,j] * A[n,k] * B[n,l];
@tullio Cx4[j,i] := A[n,i] * B[n,j] * x[n];

reshape(C4, size(C)) ≈ C
reshape(Cx4, size(Cx)) ≈ Cx

function alt!(C,Cx,A,B)
    C4 = reshape(C, M,M,M,M) # writing into original C
    Cx4 = reshape(Cx, M,M)
    @tullio C4[j,i,l,k] = A[n,i] * B[n,j] * A[n,k] * B[n,l]
    @tullio Cx4[j,i] = A[n,i] * B[n,j] * x[n]
    nothing
end

```

---

<div class="post-metadata">

**Author:** ![Germinator](https://avatars.discourse-cdn.com/v4/letter/g/3be4f8/32.png) [@Germinator](https://discourse.julialang.org/u/Germinator)\
**Post date:** [October 31, 2021, 7:52pm UTC](https://discourse.julialang.org/t/parallelization-of-for-loop/70557/4 "2021-10-31T19:52:45Z")

</div>

Amazing, thanks a lot!🙂