# Y.A.t.Q : Yet Another @threads Question

**URL:** <https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541>\
**Category:** General Usage\
**Tags:** parallel, multithreading, threads\
**Created:** [November 16, 2021, 7:56am UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541 "2021-11-16T07:56:02Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![kadir-gunel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kadir-gunel/32/35790_2.png) [@kadir-gunel](https://discourse.julialang.org/u/kadir-gunel)\
**Post date:** [November 16, 2021, 7:56am UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541/1 "2021-11-16T07:56:02Z")

</div>

Hello,

I am calculating the cosine distance between two vector spaces by using Distances.jl package.  
Since the vector spaces in comparison are large, I wanted to calculate cosine distances by using @threads. The problem is sequential for loop gives results faster than threaded version. I know that opening and closing threads might take time which in turn this might be the reason. But could someone read the threaded version, and could tell me the proper way of writing parallelized for loops in Julia ?

For minimal working example :

```nohighlight
using Base.Threads
using Distances

X = rand(Int(200e3), 300);
X_sub = X[1:4000, :];

function sequenceCosine(X4k::Matrix, X::Matrix)
    results = []
    for i in 1:size(X4k, 1)
        dist = Float64[]
        for j in 1:size(X, 1)
            push!(dist, cosine_dist(X4k[i, :], X[j, :]))
        end
        push!(results, dist)
    end
    return results
end

function parallelCosine(X_sub::Matrix, X::Matrix)
    r_threads = [Array{Float64}[] for i in 1:nthreads()];
    T = div(size(X_sub, 1), nthreads()) # got 32 threads 
    for k in 0:(T - 1)
        @threads for i in 1:nthreads()
            dist = Float64[];
            for j in 1:size(X, 1)
                push!(dist, cosine_dist(X_sub[i + (nthreads() * k), :], X[j, :]));
            end
            push!(r_threads[threadid()], dist)
        end
    end
    r_threads; # returns
end

@time result = sequenceCosine(X_sub, X); 
@time result = parallelCosine(X_sub, X); # very similar time r.t. sequence version.

```

B.R.

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [November 16, 2021, 10:39am UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541/2 "2021-11-16T10:39:09Z")

</div>

The slightly reworked MWE

```julia
using Base.Threads
using Distances
using BenchmarkTools

function sequenceCosine(X_sub::Matrix, X::Matrix)
    results = Matrix(undef, size(X_sub, 1), size(X, 1))
    for i in 1:size(X_sub, 1)
        for j in 1:size(X, 1)
            results[i, j] = cosine_dist((@view X_sub[i, :]), (@view X[j, :]))
        end
    end
    results
end

function parallelCosine(X_sub::Matrix, X::Matrix)
    results = Matrix(undef, size(X_sub, 1), size(X, 1))
    len = div(size(X_sub, 1), nthreads() - 1)
    @threads for t in 1:nthreads()
        for i in 1 + (t - 1) * len:min(t * len, size(X_sub, 1))
            for j in 1:size(X, 1)
                results[i, j] = cosine_dist((@view X_sub[i, :]), (@view X[j, :]))
            end
        end
    end
    results
end

X = rand(Int(200e2), 300);
X_sub = X[1:400, :];

result1 = sequenceCosine(X_sub, X);
result2 = parallelCosine(X_sub, X);
@assert result1 == result2

@btime result = sequenceCosine($X_sub, $X);
@btime result = parallelCosine($X_sub, $X);

```

shows

```julia
  4.429 s (8000002 allocations: 183.11 MiB)
  895.453 ms (8000034 allocations: 183.11 MiB)

```

for 6 threads on Julia 1.6.3.

---

<div class="post-metadata">

**Author:** ![kadir-gunel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kadir-gunel/32/35790_2.png) [@kadir-gunel](https://discourse.julialang.org/u/kadir-gunel)\
**Post date:** [November 16, 2021, 11:07am UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541/3 "2021-11-16T11:07:49Z")

</div>

Thank you very much for reviewing code and making it better.

I have 2 question about your answer:

1. From other working examples, I saw that in order to hold the values of each thread it is needed to create a dynamic array (` results` for my case) . And when the results are ready, they are pushed inside the that dynamic array by using the `threadid()`. From your code, I understand that there is no need for that. Could you explain the inner workings ?
2. About @view inside innermost loop. Why do we need it ?

B.R.

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [November 16, 2021, 11:15am UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541/4 "2021-11-16T11:15:52Z")

</div>

Hi,

To point 1: in my understanding you concurrently can read non changing parts of memory and write non overlapping parts of memory without data races. Therefore you only need to slice the task accordingly. In short: you have to avoid data races.

To point 2: `@view` constructs a view into an array without copying. We can use it to avoid memory allocations when reading parts of arrays.

All the best,

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [November 16, 2021, 11:24am UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541/5 "2021-11-16T11:24:37Z")

</div>

> [@goerch](#):
>
> `results = Matrix(undef, size(X_sub, 1), size(X, 1))`

This creates a `Matrix{Any}`, that doesn’t seem like a good idea.

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [November 16, 2021, 11:30am UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541/6 "2021-11-16T11:30:39Z")

</div>

Sorry for the carelessness on my part. But I didn’t notice obvious problems problems using the profiler. To OP:

```julia
Matrix{Float64}(undef, size(X_sub, 1), size(X, 1))

```

will eliminate nearly all allocations. Here are my results with this change:

```julia
  4.370 s (2 allocations: 61.04 MiB)
  1.020 s (33 allocations: 61.04 MiB)

```

---

<div class="post-metadata">

**Author:** ![kadir-gunel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kadir-gunel/32/35790_2.png) [@kadir-gunel](https://discourse.julialang.org/u/kadir-gunel)\
**Post date:** [November 16, 2021, 11:33am UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541/7 "2021-11-16T11:33:04Z")

</div>

@DNF , @goerch I thought that might be negligible but it seems like not 🙂  
Thank you both!

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [November 16, 2021, 11:35am UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541/8 "2021-11-16T11:35:12Z")

</div>

BTW, you don’t need to keep track of the threads yourself:

```julia
function pcosine(X_sub, X)
    res = similar(X, size(X_sub, 1), size(X, 1))
    @threads for j in axes(X, 1)
        for i in axes(X_sub, 1)
            res[i, j] = @views cosine_dist(X_sub[i, :], X[j, :])
        end
    end
    return res
end

```

> [@kadir-gunel](#):
>
> I thought that might be negligible

Definitely not 😉 Always avoid things like

> [@kadir-gunel](#):
>
> `results = []`

which is a red performance flag, and pre-allocating is better than incrementally `push!`ing.

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [November 16, 2021, 11:42am UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541/9 "2021-11-16T11:42:27Z")

</div>

Beautiful fast code:

```julia
function sequenceCosine(X_sub::Matrix, X::Matrix)
    results = similar(X, size(X_sub, 1), size(X, 1))
    for i in axes(X_sub, 1)
        for j in axes(X, 1)
            results[i, j] = @views cosine_dist(X_sub[i, :], X[j, :])
        end
    end
    results
end

function parallelCosine(X_sub::Matrix, X::Matrix)
    results = similar(X, size(X_sub, 1), size(X, 1))
    @threads for j in axes(X, 1)
        for i in axes(X_sub, 1)
            results[i, j] = @views cosine_dist(X_sub[i, :], X[j, :])
        end
    end
    results
end

```

with result

```julia
  4.377 s (2 allocations: 61.04 MiB)
  604.920 ms (33 allocations: 61.04 MiB)

```

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [November 16, 2021, 11:50am UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541/10 "2021-11-16T11:50:31Z")

</div>

Which leads to the following question: how to parallelize the corresponding comprehension

```julia
function sequenceCosine(X_sub::Matrix, X::Matrix)
    [@views cosine_dist(X_sub[i, :], X[j, :])
        for i in axes(X_sub, 1),
            j in axes(X, 1)]
end

```

which BTW runs faster than the looping version:

```julia
  2.739 s (2 allocations: 61.04 MiB)

```

?

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [November 16, 2021, 11:50am UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541/11 "2021-11-16T11:50:45Z")

</div>

> [@goerch](#):
>
> `results = similar(X, size(X_sub, 1), size(X, 1))`

There’s actually a tiny trap here. `similar` will fail if the inputs have an integer eltype. So to really make this robust, one probably needs to do some float promotion or something.

In fact, a parallel comprehension is probably the best way, but not sure how to do that.

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [November 16, 2021, 11:58am UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541/12 "2021-11-16T11:58:47Z")

</div>

Seems this is the only way out at the moment:

```julia
function sequenceCosine(X_sub::Matrix, X::Matrix)
    [@views cosine_dist(X_sub[i, :], X[j, :])
        for i in axes(X_sub, 1),
            j in axes(X, 1)]
end

function parallelCosine(X_sub::Matrix, X::Matrix)
    results = similar(X, size(X_sub, 1), size(X, 1))
    @threads for j in axes(X, 1)
        @views results[:, j] .=
            [cosine_dist(X_sub[i, :], X[j, :])
                for i in axes(X_sub, 1)]
    end
    results
end

```

Results:

```julia
  2.739 s (2 allocations: 61.04 MiB)
  570.136 ms (20033 allocations: 124.51 MiB)

```

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [November 16, 2021, 3:48pm UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541/13 "2021-11-16T15:48:37Z")

</div>

If you care for performance of array traversal code, it’s crucial to consider memory layout. In Julia, arrays are column-major by default, and switching to row-major gives almost a 5-fold speedup for me. This is without paralellization:

```julia
X_sub_r = PermutedDimsArray(permutedims(X_sub), (2, 1))
X_r = PermutedDimsArray(permutedims(X), (2, 1))
sequenceCosine(X_sub, X) # 3.6 s
sequenceCosine(X_sub_r, X_r) # 0.72 s

```

Writing this function in terms of `map` can make it absolutely trivial to parallelize:

```julia
# sequential - has the same performance as sequenceCosine, 0.7 s for row-major arrays
function sequenceCosine3(X_sub, X)
	map(Iterators.product(eachrow(X_sub), eachrow(X))) do (i, j)
    	cosine_dist(i, j)
	end
end

# parallel - 0.26 s for me
function parallelCosine3(X_sub, X)
	ThreadsX.map(Iterators.product(eachrow(X_sub), eachrow(X))) do (i, j)
    	cosine_dist(i, j)
	end
end

```

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [November 16, 2021, 5:56pm UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541/14 "2021-11-16T17:56:15Z")

</div>

Very nice solution! Put into my test bench and got

```julia
  2.740 s (2 allocations: 61.04 MiB)
  590.554 ms (20033 allocations: 124.51 MiB)

```

for

```julia
@btime result = sequenceCosine($X_sub, $X);
@btime result = parallelCosine($X_sub, $X);

```

and

```julia
  616.712 ms (2 allocations: 61.04 MiB)
  186.552 ms (4185 allocations: 320.18 MiB)

```

for

```julia
X_sub_r = PermutedDimsArray(permutedims(X_sub), (2, 1))
X_r = PermutedDimsArray(permutedims(X), (2, 1))

@btime result = sequenceCosine($X_sub_r, $X_r);
@btime result = parallelCosine($X_sub_r, $X_r);

```

---

<div class="post-metadata">

**Author:** ![kadir-gunel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kadir-gunel/32/35790_2.png) [@kadir-gunel](https://discourse.julialang.org/u/kadir-gunel)\
**Post date:** [November 18, 2021, 6:01am UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541/15 "2021-11-18T06:01:15Z")

</div>

Hello @aplavin ,

I was reading your post and noticed that you mentioned the row major operation is more performant than column major. Shouldn’t be vice versa ?

The ` PermuteDimsArray` with ` permutedims(X)` gives the same result as the original matrix.

```nohighlight
using Test
X = rand(10, 20);
X_r = PermuteDimsArray(permutedims(X), (2, 1))

@test X == X_r # gives true 

```

Could not catch the point. Could you please explain ?

B.R.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [November 18, 2021, 6:30am UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541/16 "2021-11-18T06:30:14Z")

</div>

I can’t believe I missed that you were working row major, that’s of course an important difference.

This

> [@kadir-gunel](#):
>
> `X_r = PermuteDimsArray(permutedims(X), (2, 1))`

is a somewhat obscure way of making `X_r` into a row major array. `permutedims` transposes `X`, and `PermureDimsArray` transposes it back again, except lazily. So `X_r` is now the same as `X` except it’s row major, leading to a significant speedup.

But for you, @kadir-gunel, it would be better if you organize your problem so that the matrices hold the interesting data along their columns in the first place. Then you can rewrite the function to work along columns.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [November 18, 2021, 6:58am UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541/17 "2021-11-18T06:58:23Z")

</div>

> [@kadir-gunel](#):
>
> the row major operation is more performant than column major. Shouldn’t be vice versa ?

And, in case this isn’t clear, your code works along rows, therefore working on a row-major array is more efficient than on one that is column-major

---

<div class="post-metadata">

**Author:** ![kadir-gunel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kadir-gunel/32/35790_2.png) [@kadir-gunel](https://discourse.julialang.org/u/kadir-gunel)\
**Post date:** [November 18, 2021, 7:05am UTC](https://discourse.julialang.org/t/y-a-t-q-yet-another-threads-question/71541/18 "2021-11-18T07:05:21Z")

</div>

Yes, my bad. Thank you for clarification.