Performance of variants of findall

General Usage Performance
1

Here is a small session:

julia> using BenchmarkTools

julia> A = rand(100, 100)
100×100 Array{Float64,2}  [...]

julia> @btime findall(A .> 0.5)
  39.951 μs (7 allocations: 85.06 KiB)
5077-element Array{CartesianIndex{2},1}  [...]

julia> @btime findall(x -> x > 0.5, A)
  208.314 μs (17 allocations: 256.80 KiB)
5077-element Array{CartesianIndex{2},1} [...]

Why these timing differences between evaluations of two expressions that look so close and sensible ?

1 Like
2

They are not the same (you can explore this with @edit, or the debugger). The first creates a temporary array for A .> 0, and then collects the true indexes. Both of these operations are very fast.

The second one goes through the generic iterator path. Perhaps it could optimized more, I am sure PRs doing this would be welcome.

3

Thanks for your answer. Perhaps some warning in the documentation (Arrays · The Julia Language), about the case where the second argument is an array, would be useful.

4

Try to see if there is any effect of interpolating A into the expression with $

5

I don’t think there is an inherent reason for the performance difference here (which would of course warrant documentation), it is just waiting for someone to optimize it.

6

See also the similar topic Comprehension vs map and filter unexpected speeds

1 Like
7

Sorry I am new,
but both of these gives the same result say if i want to find the indexes where the maximum values occur, the first one is way faster…
Can you please explain this more

8 
julia> using BenchmarkTools

julia> A = rand(100, 100);

julia> @btime findall($A .> 0.5);
  9.208 μs (5 allocations: 84.36 KiB)

julia> @btime findall(x -> x > 0.5, $A);
  9.058 μs (5 allocations: 84.36 KiB)

seems same speed to me

1 Like
9 

julia> @btime findall(x->x==maximum(b),b);
  49.071 ms (29504 allocations: 462.69 KiB)

julia> @btime findall(x->x==maximum(b),$b);
  48.989 ms (29504 allocations: 462.69 KiB)

julia> @btime findall(b.==maximum(b))
  9.910 μs (7 allocations: 5.72 KiB)

but when trying to find maximum there is a big difference.

10

try this:

@btime let maxval=maximum($b); findall(x->x==maxval,$b); end;
11

outside btime i can’t use $ it shows,

ERROR: syntax: "$" expression outside quote around REPL[187]:1
Stacktrace:
 [1] top-level scope
   @ REPL[187]:1
12

Don’t use $ outside btime.

13

Can you explain me more about this error…?
And the usage of $

14

what’s the point of $ if i can’t use it anywhere else except while benchmarking…

15

For every iteration, this calculates the maximum again and again. If there are 100x100 element, it will pass over the array 10,000 times. You only need to calculate the maximum once.

1 Like
16

$ is used for string interpolation and code expression interpolation—allowing you to splice a variable directly into a string or code expression. For example:

julia> x="world"   # string
       "hello $x"  # string interpolation
"hello world"

julia> y=:( a += 1 )        # expression
       :( if true; $y end ) # expression interpolation
:(if true
      #= REPL[2]:2 =#
      a += 1
  end)

Because @btime is a macro, it operates on an expression. The authors of BenchmarkTools decided that $ would capture the variable from the environment and interpolate it into the expression, while passing through type information and blocking constant propagation.

All that to say: when you’re building strings and expressions, and sometimes when calling macros, $ can be useful. But for normal code, $ shouldn’t be dangling around.

2 Likes