Simple, understandable iterator

General Usage Performance
,
1

It’s been noted that a lot of Julia code has more unnecessary allocation and construction of containers than, say, Rust code. I think this is due in large part to Julia’s dual role as a programming language and an application (I probably mentioned this here a while ago.) It’s a bit inconvenient at the REPL to wrap a returned value in collect (after you try to index into it)

Still, I’m trying to do a bit more of this. I’m looking for easy ways to create lazy collections. There are a lot of packages that might help. Generators are easy.

julia> const v1 = fill(1, 100);
julia> get_numbers1(v) = (float(x) for x in v);

julia> get_numbers1(v1)
Base.Generator{Vector{Int64}, var"#3#4"}(var"#3#4"(), [1, 1, 1, 1, 1, 1, 1, 1, 1, 1  …  1, 1, 1, 1, 1, 1, 1, 1, 1, 1])

But as a user, especially a new one, the type of this iterable is confusing, and in fact you can’t see that it returns Float64s.

This is a little better.

julia> get_numbers2(v) = Base.Generator(float, v);

julia> it2 = get_numbers2(v1)
Base.Generator{Vector{Int64}, typeof(float)}(float, [1, 1, 1, 1, 1, 1, 1, 1, 1, 1  …  1, 1, 1, 1, 1, 1, 1, 1, 1, 1])

You can look at the docstring for Generator to find out what’s happening. But it still has a lot of visual noise. And if I nest a couple of these, the type gets even harder to read. Annotating the return type, or documenting it, is difficult (this happens in Rust, too, but an end user doesn’t typically see it)

I’m considering trying a wrapper:

struct Iter{T,GT}
    iter::GT
end
@inline Iter{T}(iter) where T = Iter{T, typeof(iter)}(iter)
Base.print(io::IO, ::Iter{T}) where {T} = print(io, "Iter{", T, "}")
Base.show(io::IO, ::MIME"text/plain", iter::Iter) = print(io, iter)
@inline Base.iterate(it::Iter, args...) = iterate(it.iter, args...)
@inline Base.length(it::Iter) = length(it.iter)
@inline Base.eltype(::Iter{T}) where {T} = T

Then

julia> get_numbers3(v) = Iter{Float64}(get_numbers1(v));

julia> it3 = get_numbers3()
Iter{Float64}

It’s obvious what this thing returns. They all have equal performance

julia> @btime sum(it3);
  59.225 ns (0 allocations: 0 bytes)

But it’s not very good

julia> @btime sum(float, v1);
  7.013 ns (0 allocations: 0 bytes)

I wonder what kinds of ideas are out there?

4 Likes