# Is boolean indexing 100 times slower in 1.0?

**URL:** <https://discourse.julialang.org/t/is-boolean-indexing-100-times-slower-in-1-0/16286>\
**Category:** Performance\
**Created:** [October 13, 2018, 6:34pm UTC](https://discourse.julialang.org/t/is-boolean-indexing-100-times-slower-in-1-0/16286 "2018-10-13T18:34:08Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![misha\_mikhasenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/misha_mikhasenko/32/5060_2.png) [@misha\_mikhasenko](https://discourse.julialang.org/u/misha_mikhasenko)\
**Post date:** [October 13, 2018, 6:34pm UTC](https://discourse.julialang.org/t/is-boolean-indexing-100-times-slower-in-1-0/16286/1 "2018-10-13T18:34:08Z")

</div>

```julia
d = rand(100)
mask = [fill(false,20)...,fill(true,80)...];
@time for i in 1:100000
    d[mask]
end

```

- Julia-1.0 and 0.7

```julia
  3.038767 seconds (41.20 M allocations: 1.197 GiB, 4.90% gc time)

```

- Julia-0.6.4

```julia
  0.029932 seconds (300.00 k allocations: 74.768 MiB, 11.41% gc time)

```

---

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [October 13, 2018, 6:49pm UTC](https://discourse.julialang.org/t/is-boolean-indexing-100-times-slower-in-1-0/16286/2 "2018-10-13T18:49:46Z")

</div>

Don’t benchmark in global scope, but the conclusion remains the same:

```julia
d = rand(100)
mask = [fill(false,20)...,fill(true,80)...]
using BenchmarkTools
@btime $d[$mask]

```

22.229 μs on 1.0, 148.242 ns on 0.6.

See also [Performance of logical indexing in Julia 0.7 · Issue #29418 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/29418).

---

<div class="post-metadata">

**Author:** ![misha\_mikhasenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/misha_mikhasenko/32/5060_2.png) [@misha\_mikhasenko](https://discourse.julialang.org/u/misha_mikhasenko)\
**Post date:** [October 13, 2018, 6:58pm UTC](https://discourse.julialang.org/t/is-boolean-indexing-100-times-slower-in-1-0/16286/3 "2018-10-13T18:58:11Z")

</div>

> [@tkoolen](#):
>
> Don’t benchmark in global scope

good point, thanks.

---

<div class="post-metadata">

**Author:** ![aaowens](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aaowens/32/12101_2.png) [@aaowens](https://discourse.julialang.org/u/aaowens)\
**Post date:** [October 13, 2018, 7:07pm UTC](https://discourse.julialang.org/t/is-boolean-indexing-100-times-slower-in-1-0/16286/4 "2018-10-13T19:07:06Z")

</div>

For reference the comprehension version is fast in 1.0

```julia
julia> function test1(d, mask)
       d[mask]
       end
test1 (generic function with 1 method)

julia> function test2(d, mask)
       [d[i] for i in eachindex(d) if mask[i]]
       end
test2 (generic function with 1 method)

julia> @btime test1($d, $mask)
  27.081 μs (412 allocations: 12.55 KiB)
80-element Array{Float64,1}:
 0.8054772608752572  
 0.05067482842558224 
 0.5388726134765767  
 0.692159153505675   
 0.7684311492067795  
 0.6711078136106581  
 0.6649141807497929  
 0.5985228802273888  
 0.3748395020447204  
 0.914509451795469   
 ⋮                   
 0.4337689060373162  
 0.793914486706204   
 0.5254852110421662  
 0.37372068234717726 
 0.9288662798362115  
 0.495094572109253   
 0.21356511652770882 
 0.022123152385092437
 0.2705493240621617  

julia> @btime test2($d, $mask)
  958.000 ns (12 allocations: 2.44 KiB)
80-element Array{Float64,1}:
 0.8054772608752572  
 0.05067482842558224 
 0.5388726134765767  
 0.692159153505675   
 0.7684311492067795  
 0.6711078136106581  
 0.6649141807497929  
 0.5985228802273888  
 0.3748395020447204  
 0.914509451795469   
 ⋮                   
 0.4337689060373162  
 0.793914486706204   
 0.5254852110421662  
 0.37372068234717726 
 0.9288662798362115  
 0.495094572109253   
 0.21356511652770882 
 0.022123152385092437
 0.2705493240621617  

```

---

<div class="post-metadata">

**Author:** ![Ralph\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ralph_smith/32/10344_2.png) [@Ralph\_Smith](https://discourse.julialang.org/u/Ralph_Smith)\
**Post date:** [October 13, 2018, 7:07pm UTC](https://discourse.julialang.org/t/is-boolean-indexing-100-times-slower-in-1-0/16286/5 "2018-10-13T19:07:37Z")

</div>

Try converting your mask to a `BitArray`, which has an optimized iterator.

---

<div class="post-metadata">

**Author:** ![misha\_mikhasenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/misha_mikhasenko/32/5060_2.png) [@misha\_mikhasenko](https://discourse.julialang.org/u/misha_mikhasenko)\
**Post date:** [October 13, 2018, 7:30pm UTC](https://discourse.julialang.org/t/is-boolean-indexing-100-times-slower-in-1-0/16286/6 "2018-10-13T19:30:20Z")

</div>

I realized that for my case the boolean indexing is not necessary. I could use an array of indexes.

```julia
d = rand(100)
mask = [fill(false,20)...,fill(true,80)...]
inds = collect(1:length(d))[mask];
@btime $d[$inds]

```

is back to right speed for 1.0

```julia
134.743 ns (2 allocations: 752 bytes)

```

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [October 13, 2018, 8:38pm UTC](https://discourse.julialang.org/t/is-boolean-indexing-100-times-slower-in-1-0/16286/8 "2018-10-13T20:38:20Z")

</div>

You can fix the problem by the following:

```julia
julia> using BenchmarkTools

julia> d=rand(10_000); m=rand(Bool, 10_000);

julia> @btime getindex($d,$m);
  742.062 μs (23906 allocations: 643.13 KiB)

julia> Base.@propagate_inbounds function Base.iterate(L::Base.LogicalIndex, s)
           # We're looking for the n-th true element, using iterator r at state i
           n = s[1]
           n > length(L) && return nothing
           idx, i = iterate(Base.tail(s)...)
           s = (n+1, s[2], i)
           L.mask[idx] && return (idx, s)

           
           while true
               idx, i = iterate(Base.tail(s)...)
               s = (n+1, s[2], i)
               L.mask[idx] && return (idx, s)
           end
           end

julia> @btime getindex($d,$m);
  49.477 μs (4 allocations: 38.63 KiB)

```

---

<div class="post-metadata">

**Author:** ![misha\_mikhasenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/misha_mikhasenko/32/5060_2.png) [@misha\_mikhasenko](https://discourse.julialang.org/u/misha_mikhasenko)\
**Post date:** [October 14, 2018, 10:02am UTC](https://discourse.julialang.org/t/is-boolean-indexing-100-times-slower-in-1-0/16286/9 "2018-10-14T10:02:49Z")

</div>

@foobar_lv2, thank you for clarifying the problem,

> [@foobar\_lv2](#):
>
> (post withdrawn by author, will be automatically deleted in 24 hours unless flagged)

What do you mean? Do you suggest to flag it or withdraw? I have not solved the problem just found a workaround. I guess it is a bug to be fixed.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [October 14, 2018, 11:14am UTC](https://discourse.julialang.org/t/is-boolean-indexing-100-times-slower-in-1-0/16286/10 "2018-10-14T11:14:25Z")

</div>

> [@misha\_mikhasenko](#):
>
> Do you suggest to flag it or withdraw?

This is just a message that appears from Discourse when people withdraw _their own_ posts.

---

<div class="post-metadata">

**Author:** ![misha\_mikhasenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/misha_mikhasenko/32/5060_2.png) [@misha\_mikhasenko](https://discourse.julialang.org/u/misha_mikhasenko)\
**Post date:** [October 14, 2018, 11:39am UTC](https://discourse.julialang.org/t/is-boolean-indexing-100-times-slower-in-1-0/16286/11 "2018-10-14T11:39:43Z")

</div>

That is funny how I misunderstood it. Alright, thanks.

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [October 14, 2018, 11:42am UTC](https://discourse.julialang.org/t/is-boolean-indexing-100-times-slower-in-1-0/16286/12 "2018-10-14T11:42:38Z")

</div>

I first posted a completely wrong explanation, so I withdrew the bad post (my fault). Once I had the proper explanation, I made the work-around and submitted it as a PR.

The explanation I did not post here, btw, is that i`terate(L::LogicalIndex{Int})` was not correctly inferred, because the state `s` can change type after the first iteration in the above loop. Unrolling the first iteration fixes that.