Inconsistency in `sum` of an empty generator

Internals & Design
,
1

On Julia 1.3, the following two empty generators both throw an error when you apply sum to them:

julia> g1 = (1 for i in Int[]);

julia> g2 = (x for x in [-1] if x > 0);

julia> eltype(g1)
Any

julia> eltype(g2)
Any

julia> sum(g1)
ERROR: ArgumentError: reducing over an empty collection is not allowed

julia> sum(g2)
ERROR: ArgumentError: reducing over an empty collection is not allowed

That seems like the correct behavior. On newer versions of Julia, you can avoid the error by setting the init keyword argument. (Not sure which version introduced that keyword argument.)

However, on Julia 1.4 the second generator sums to zero:

julia> g1 = (1 for i in Int[]);

julia> g2 = (x for x in [-1] if x > 0);

julia> eltype(g1)
Any

julia> eltype(g2)
Any

julia> sum(g1)
ERROR: ArgumentError: reducing over an empty collection is not allowed

julia> sum(g2)
0

Was that an intended change or an accidental change?

1 Like
2

No idea about the inconsistency highlighted, but I just wanted to point out the fact that for the empty generator g2, Julia’s results:

sum(g2)   # = 0
prod(g2)  # = 1

seem to be consistent with the definitions of the empty sum and empty product.

2 Likes
3

As far as I can tell, sum(g) does not promise to be exactly equivalent to reduce(+, g) so I do not have a problem with sum(g2) working.

What seems more concerning is that reduce(+, Number[]) also returns 0. Clearly the error message is incorrect here when it implies reducing over an empty collection is never allowed.

4

Reducing over an empty collection is not allowed if the eltype cannot be determined, because then it is impossible to know the correct type of 0 (for sum).

The reason it works for g2 is that eltype(g2.iter) is known (== Int), and that is what is used nowadays under the hood for filtered iterators. No idea why g1 can’t use eltype(g1.iter), though — it seems like that could be improved.

4 Likes
5

Oh, that’s interesting. Why does eltype(g2) == Any, but eltype(g2.iter) == Int?

6

I see. The iter field of a generator is the iter in the following expression:

g = (f(x) for x in iter)

A generator has two fields, f and iter. It appears that there is a specialization for the case when f == identity:

julia> sum(x for x in Int[])
0

julia> sum(2x for x in Int[])
ERROR: MethodError: reducing over an empty collection is not
allowed; consider supplying `init` to the reducer

I guess it makes sense, because the generator (x for x in iter) should be exactly the same as just iterating through iter.

This explains why g1 doesn’t work: It’s because x -> 1 is not the identity function. sum will throw an error when f is not the identity function because the return type of f might be different from the element type of iter.

3 Likes