Fixing the Piping/Chaining Issue

Internals & Design
, , , ,
162

A bit more compile-time benchmarking…

# State of the art, no piping:
julia> @btime eval(:(  map(Base.Fix2(^,2), filter(isodd, [1,2,3]))  )) 
  91.500 μs (65 allocations: 3.42 KiB)
2-element Vector{Int64}:
 1
 9

# OP proposal:
julia> @btime eval(:(  FixFirst(map, FixLast(^,2))(FixFirst(filter, isodd)([1,2,3]))  )) 
  93.600 μs (70 allocations: 3.70 KiB)
2-element Vector{Int64}:
 1
 9

# SOTA + piping
julia> @btime eval(:(  [1,2,3] |> Base.Fix1(filter, isodd) |> Base.Fix1(map, Base.Fix2(^,2))  )) 
  125.600 μs (91 allocations: 4.77 KiB)
2-element Vector{Int64}:
 1
 9

# OP proposal for partial functions, but using piping
julia> @btime eval(:(  [1,2,3] |> FixFirst(filter, isodd) |> FixFirst(map, FixLast(^,2))  )) 
  95.700 μs (70 allocations: 3.80 KiB)
2-element Vector{Int64}:
 1
 9

# New proposal for general-purpose functor w/ piping (à la #24990)
julia> @btime eval(:(  [1,2,3] |> Fix{(1,)}(filter,(isodd,)) |> Fix{(1,)}(map,(Fix{(2,)}(^,(2,)),))  )) 
  170.400 μs (146 allocations: 7.31 KiB)
2-element Vector{Int64}:
 1
 9

# piping into anonymous functions
julia> @btime eval(:(  [1,2,3] |> x->filter(isodd, x) |> x->map(Base.Fix2(^,2), x)  )) 
  3.439 ms (6780 allocations: 387.51 KiB)
2-element Vector{Int64}:
 1
 9

# one anonymous function, fed to `filter`, no pipes
julia> @btime eval(:(  map(Base.Fix2(^,2), filter(x->x%2==1, [1,2,3]))  ))
  8.452 ms (11000 allocations: 567.00 KiB)
2-element Vector{Int64}:
 1
 9

# one anonymous function, fed to `map`, no pipes
julia> @btime eval(:(  map(x->x^2, filter(isodd, [1,2,3]))  )) 
  19.296 ms (48195 allocations: 2.47 MiB)
2-element Vector{Int64}:
 1
 9

# `map` and `filter` on anonymous functions, no pipes
julia> @btime eval(:(  map(x->x^2, filter(x->x%2==1, [1,2,3]))  ))
  30.295 ms (59130 allocations: 3.02 MiB)
2-element Vector{Int64}:
 1
 9

# all anonymous functions w/ pipes
julia> @btime eval(:(  [1,2,3] |> x->filter(x->x%2==1, x) |> x->map(x->x^2, x)  ))
  29.688 ms (64845 allocations: 3.34 MiB)
2-element Vector{Int64}:
 1
 9

Look at those last four! :scream: Why’s it so slow!?

@dlakelan looks like I owe you an apology, it appears anonymous functions can take glacial amounts of time to compile sometimes (not sure when or why yet). Gives huge motivation to use partial application functors (e.g. Fix) instead where possible.

Edit: there’s more nuance to compile time, see below.

1 Like