# Expected 72X speedup, observed 1.1X

**URL:** <https://discourse.julialang.org/t/expected-72x-speedup-observed-1-1x/60071>\
**Category:** Performance\
**Created:** [April 27, 2021, 12:58am UTC](https://discourse.julialang.org/t/expected-72x-speedup-observed-1-1x/60071 "2021-04-27T00:58:42Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![PhilipDLoewen](https://avatars.discourse-cdn.com/v4/letter/p/dec6dc/32.png) [@PhilipDLoewen](https://discourse.julialang.org/u/PhilipDLoewen)\
**Post date:** [April 27, 2021, 12:58am UTC](https://discourse.julialang.org/t/expected-72x-speedup-observed-1-1x/60071/1 "2021-04-27T00:58:42Z")

</div>

I dreamed up a simple-looking task that I thought multithreading could accelerate. Running the file below (I named it `threadbench.jl`) proved me wrong: with 72 cores and 72 threads, the timings are not appreciably better than with a single thread. Understanding why I don’t get 72X speedup may help me in my long journey to improving some code I actually care about. Any ideas?

```julia
using Base.Threads

# This function does the same simple job either with or without threading.
function tt(invec; threadit=false)
	outvec = zeros(size(invec))
	if threadit
		@threads for i in 1:size(invec,1)
			outvec[i] = threadid()+invec[i]
		end
	else
		for i in 1:size(invec,1)
			outvec[i] = threadid()+invec[i]
		end
	end
	return outvec
end

# Run the function once to make sure it's compiled
z = rand(1024)
y = tt(z)
y = tt(z,threadit=true) # Exercise all branches to be extra sure

x = round.(rand(2^27),digits=2)
	
println("Timing with one thread:")
@time y1 = tt(x,threadit=false)
	
println("Timing with $(Threads.nthreads()) threads:")
@time y72 = tt(x,threadit=true)

```

On my machine, the results are as follows:

```julia
Timing with one thread:
  1.443072 seconds (2 allocations: 1.000 GiB, 3.54% gc time)
Timing with 72 threads:
  1.263002 seconds (367 allocations: 1.000 GiB)

```

Thanks!

---

<div class="post-metadata">

**Author:** ![affans](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/affans/32/11911_2.png) [@affans](https://discourse.julialang.org/u/affans)\
**Post date:** [April 27, 2021, 1:19am UTC](https://discourse.julialang.org/t/expected-72x-speedup-observed-1-1x/60071/2 "2021-04-27T01:19:52Z")

</div>

```julia
julia> @time y1 = tt(x,threadit=false);
  0.252025 seconds (2 allocations: 1.000 GiB, 0.79% gc time)

julia> @time y72 = tt(x,threadit=true);
  0.227822 seconds (45 allocations: 1.000 GiB, 1.30% gc time)

```

with 8 threads on my i7 laptop. I have these speeds because I declared `x` to be `const`. What I am noticing is that the times jump wildly between 0.25 - 0.45 seconds for me in both scenarios.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [April 27, 2021, 1:24am UTC](https://discourse.julialang.org/t/expected-72x-speedup-observed-1-1x/60071/3 "2021-04-27T01:24:58Z")

</div>

> [@PhilipDLoewen](#):
>
> ```julia
> @threads for i in 1:size(invec,1)
> outvec[i] = threadid()+invec[i]
> end
> 
> ```

This code basically spends all its time reading/writing memory (the actual addition should be negligible), and your machine probably doesn’t have the memory bandwidth for all your cores to access that much RAM simultaneously (the 1GiB `invec` and `outvec` arrays are too big for the caches).

Try something compute-bound, not memory-bound.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [April 27, 2021, 1:26am UTC](https://discourse.julialang.org/t/expected-72x-speedup-observed-1-1x/60071/4 "2021-04-27T01:26:31Z")

</div>

> [@affans](#):
>
> What I am noticing is that the times jump wildly between 0.25 - 0.45 seconds for me in both scenarios.

Use `@btime tt($x,threadit=true);` from the [BenchmarkTools.jl package](https://github.com/JuliaCI/BenchmarkTools.jl) instead of `@time`. It will take several measurements, so it will be less noisy, and the `$x` means that you don’t need to declare `x` as `const` to avoid a global-variable penalty.

---

<div class="post-metadata">

**Author:** ![PhilipDLoewen](https://avatars.discourse-cdn.com/v4/letter/p/dec6dc/32.png) [@PhilipDLoewen](https://discourse.julialang.org/u/PhilipDLoewen)\
**Post date:** [April 28, 2021, 4:36am UTC](https://discourse.julialang.org/t/expected-72x-speedup-observed-1-1x/60071/5 "2021-04-28T04:36:05Z")

</div>

Thanks a lot, both of you, for taking the time to reply. I will continue on my journey of learning and exploration, hoping that the next issue I encounter will be worthy of starting a new thread (not continuing here).

I took Steven’s advice and changed the task to be split into @threads: the new job was to compute a large number of random N-by-N determinants (with N=5 or N=3). At first I used the `LinearAlgebra.jl` package for this, and managed to crash BLAS: this was surprising / gratifying / educational! (Apparently OPENBLAS does its own threading, and the collision between that and my explicit threading requests was too much to handle.) Specifying OPENBLAS\_NUM\_THREADS=1 before starting Julia overcame this issue, but nonetheless the version of my task using @threads was significantly _slower_ than the single-threaded version. Finally I wrote my own super-simple 3-by-3 determinant evaluator function, and then the threading approach allowed a modest speedup – around 6X. Perhaps memory access is still the bottleneck, since 10 Julia threads are just as fast as 72 for this operation.

There is no new question here, just a little story to confirm that I took your advice seriously and learned from it. Perhaps some future reader will, too. Thanks again!

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [April 28, 2021, 4:50am UTC](https://discourse.julialang.org/t/expected-72x-speedup-observed-1-1x/60071/6 "2021-04-28T04:50:41Z")

</div>

Note that if you’re working with small arrays of fixed sizes, (smaller than `10x10` or so), `StaticArrays` will often be much faster since it stores the size of the array in the type, and therefore can generate specialized code.