`Nulls.skip` is very slow

Specific Domains Data
1

Please excuse how rough this example is (but I think it is essentially correct)

using Nulls
using BenchmarkTools
      
const N = 10^7
const n = 10^5

A = convert(Vector{Union{Int64,Null}}, rand(Int64, N))
                               
A[rand(1:N, n)] = null
    
x = 0

@btime for a ∈ $A
    !isnull(a) && ($x += a)
end

x = 0

@btime for a ∈ Nulls.skip($A)
    $x += a
end

this gives

  1.163 s (39602147 allocations: 604.28 MiB)
  11.625 s (148507554 allocations: 3.25 GiB)

Thing is, the source code for Nulls.skip is dead simple, so I’m not really sure what can be done about this. I suppose this has something to do with allocating the iterator, though I never would have expected it to be quite this slow.

What’s our solution here? What should we be telling people to do in these cases? Is there really no substitute for explicit null handling?

2

Thanks for reporting this. I think that’s a case where the optimizations for Union{T, Null} have not been fully implemented in Julia 0.7 yet. Specifically, inlining of next does not happen (see this explanation by @quinnj), which kills performance for such a simpler iterator object.

However, Julia 0.7 is about twice faster than Julia 0.6 in my tests, so that’s already a good sign (of course that’s not acceptable). Until optimizations have landed, better continue to use DataArrays, which are going to be ported to Nulls soon so that the ecosystem becomes more consistent.

We should definitely add a few test cases like this to Nulls using PkgBenchmark (or even in Base, as this is a common pattern).

1 Like
3

Thanks. If anyone around is using 0.7 and can run my script on it I would be very interested to see the results. Frankly, I am extremely eager to switch everything to Nulls because I just can’t handle the inconsistency any more, so I may have jumped the gun a bit. Fortunately, the use of union types in arrays seems surprisingly fast even in 0.6 as long as one uses methods specifically written to handle them.

1 Like
4

As I said above, it’s about twice faster than 0.6 currently.

EDIT: and the version which calls isnull is 100× faster, and does almost no allocations.

5

I have found a way to work around this problem: https://github.com/JuliaData/Nulls.jl/pull/50

With that PR, I get:

julia> @btime for a ∈ $A
           !isnull(a) && ($x += a)
       end
  1.382 s (39601951 allocations: 604.28 MiB)

julia> @btime for a ∈ Nulls.skip($A)
           $x += a
       end
  66.971 ms (0 allocations: 0 bytes)
2 Likes