ANN: Transducers.jl, efficient and composable algorithms for map- and reduce-like operations

Package Announcements
7

Good point. Here is an example that is very “hostile” to iterator:

julia> xs = 1:2^15
1:32768

julia> @btime foldl(+, Iterators.flatten(1:x for x in xs))
  200.448 ms (10 allocations: 352 bytes)
5864598896640

julia> @btime mapfoldl(Map(x -> 1:x) |> Cat(), +, xs)
  45.532 μs (18 allocations: 880 bytes)
5864598896640

Piggybacking on Skipnothing is missing, here is a more reasonable case for comparison:

julia> xs = [abs(x) > 1 ? nothing : x for x in randn(2^20)];

julia> @btime foldl(+, Iterators.filter(!isnothing, xs); init=0.0)
  4.084 ms (4 allocations: 80 bytes)
779.0559461186579

julia> @btime mapfoldl(Filter(!isnothing), +, xs, init=0.0)
  2.913 ms (5 allocations: 80 bytes)
779.0559461186579
4 Likes