# For loop Performance

**URL:** <https://discourse.julialang.org/t/for-loop-performance/33214>\
**Category:** Performance\
**Tags:** question\
**Created:** [January 10, 2020, 9:10pm UTC](https://discourse.julialang.org/t/for-loop-performance/33214 "2020-01-10T21:10:46Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![sam1988](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sam1988/32/12238_2.png) [@sam1988](https://discourse.julialang.org/u/sam1988)\
**Post date:** [January 10, 2020, 9:10pm UTC](https://discourse.julialang.org/t/for-loop-performance/33214/1 "2020-01-10T21:10:46Z")

</div>

Can somebody help me in speeding up following function

function mean\_loop(p::AbstractArray,q::AbstractArray)  
a = p;  
a1 = q;  
z=0.0;  
for i in 1:length(a1)  
s=0.0;  
@inbounds s= mean(a .\> a1[i]);  
z+= s;  
s=0.0  
end  
return z  
end  
@time mean\_loop(rand(10^6),rand(10^6)) :902.126519 secs

---

<div class="post-metadata">

**Author:** ![improbable22](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/improbable22/32/5464_2.png) [@improbable22](https://discourse.julialang.org/u/improbable22)\
**Post date:** [January 10, 2020, 9:24pm UTC](https://discourse.julialang.org/t/for-loop-performance/33214/2 "2020-01-10T21:24:16Z")

</div>

This isn’t horribly slow, but I think it can be much more compact:

```julia
julia> mean_cast(p,q) = sum(p .> q') / length(p);

julia> mean_iter(p,q) = sum(p1>q1 for p1 in p, q1 in q) / length(p);

julia> using BenchmarkTools

julia> @btime mean_loop($(rand(10^4)),$(rand(10^4)));
  107.303 ms (30000 allocations: 54.17 MiB)

julia> @btime mean_cast($(rand(10^4)),$(rand(10^4)));
  128.281 ms (4 allocations: 11.93 MiB)

julia> @btime mean_iter($(rand(10^4)),$(rand(10^4)));
  76.649 ms (7 allocations: 160 bytes)

```

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [January 10, 2020, 9:33pm UTC](https://discourse.julialang.org/t/for-loop-performance/33214/3 "2020-01-10T21:33:49Z")

</div>

Try this:

```julia
function mean_loop(p::AbstractArray, q::AbstractArray)
    z = 0.0
    for x in q
        z += mean(t -> t > x, p)
    end
    return z
end

```

---

<div class="post-metadata">

**Author:** ![improbable22](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/improbable22/32/5464_2.png) [@improbable22](https://discourse.julialang.org/u/improbable22)\
**Post date:** [January 10, 2020, 9:55pm UTC](https://discourse.julialang.org/t/for-loop-performance/33214/4 "2020-01-10T21:55:41Z")

</div>

Nicely done, that’s about 7x faster than my `mean_iter`. Which is a little surprising to me, don’t they iterate through the same things?

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [January 10, 2020, 9:57pm UTC](https://discourse.julialang.org/t/for-loop-performance/33214/5 "2020-01-10T21:57:17Z")

</div>

Welcome to the JuliaLang discourse @sam1988!

I’d recommend reading [Please read: make it easier to help you](https://discourse.julialang.org/t/psa-make-it-easier-to-help-you/14757) for advice on how to format your code and ask questions in a way that makes it more likely for you to get high quality responses which are also useful to others in the future (thought it appears you’ve already got some good replies!).

Furthermore, when benchmarking your code, using `@time` can be very misleading. In the example you gave, `@time mean_loop(rand(10^6),rand(10^6))` will include the time it takes to compile the function `mean_loop` and the time it takes to create the arrays `rand(10^6)` and `rand(10^6)`, so most of the time you’re measuring is not actually code relevant to `mean_loop`. Instead, I would use the `@btime` macro from [BenchmarkTools.jl](https://github.com/JuliaCI/BenchmarkTools.jl) to properly measure performance like this:

```julia
using BenchmarkTools
@btime mean_loop($(rand(10^6)), $(rand(10^6)))

```

The `@btime` macro will not include compile time in it’s benchmarks, and it runs the benchmarking function many times so it can give statistically significant results). When I write `$(rand(10^6))` inside `@btime`, that means that I am actually precomputing the array `rand(10^6)` and pasting the value in as a constant to the function before the benchmark loop is run.

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [January 10, 2020, 10:37pm UTC](https://discourse.julialang.org/t/for-loop-performance/33214/6 "2020-01-10T22:37:32Z")

</div>

> [@improbable22](#):
>
> Which is a little surprising to me, don’t they iterate through the same things?

I guess there’s some kind of overhead to the nested loop generator. Splitting it into two generators seems faster.

```julia
mean_iter2(p, q) = sum(sum(p1 > q1 for p1 in p) for q1 in q) / length(p)

```

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [January 10, 2020, 11:37pm UTC](https://discourse.julialang.org/t/for-loop-performance/33214/7 "2020-01-10T23:37:27Z")

</div>

You can go a bit faster with integer arithmetic:

```julia
function mean_loop_int(p::AbstractArray, q::AbstractArray)
    z = 0
    for x in q
        z += count(>(x), p)
    end
    return z / length(p) 
end

```

```julia
julia> x = rand(10^4);

julia> y = rand(10^4);

julia> @btime mean_loop($x, $y)
  19.034 ms (0 allocations: 0 bytes)
4968.276699999993

julia> @btime mean_loop_int($x, $y)
  15.797 ms (0 allocations: 0 bytes)
4968.2767

```

But if you change the algorithm, you can get significant speedups:

```julia
function mean_sorted(p, q)
    ps = sort(p)
    Lq = length(q)
    z = 0
    for x in q
        ind = searchsortedfirst(ps, x)
        z += Lq - ind + 1
    end
    return z / length(p)
end

```

```julia
julia> @btime mean_sorted($x, $y)
  1.512 ms (2 allocations: 78.20 KiB)
4968.2767

```

Possibly, it can be even faster if we sort both arrays and search for the next larger at each iteration, but I couldn’t immediately find a `searchsortednext` function.

This also scales well for longer vectors. For 10^6 length I get:

```julia
julia> @btime mean_sorted(x, y) setup=(x=rand(10^6); y=rand(10^6))
  395.748 ms (2 allocations: 7.63 MiB)
500344.792228

```

That is, N \log N complexity, rather than N^2.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [January 11, 2020, 12:03am UTC](https://discourse.julialang.org/t/for-loop-performance/33214/8 "2020-01-11T00:03:08Z")

</div>

> [@DNF](#):
>
> but I couldn’t immediately find a `searchsortednext` function.

How about using `view`?

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [January 11, 2020, 12:16am UTC](https://discourse.julialang.org/t/for-loop-performance/33214/9 "2020-01-11T00:16:33Z")

</div>

Yeah, I guess.