Inconsistency with multidimensional comprehensions and iterators

General Usage
1

I found this behavior surprising:

julia> using Iterators: product

julia> p = product(1:2, 1:2)
Iterators.Product{Tuple{UnitRange{Int64},UnitRange{Int64}}}([(1:2, 1:2))

julia> [1 for x in p, y in 1:2]
8-element Array{Int64,1}:
 1
 1
 1
 1
 1
 1
 1
 1

julia> [1 for x in collect(p), y in 1:2]
4×2 Array{Int64,2}:
 1  1
 1  1
 1  1
 1  1

I expected both versions to behave the way collect(p) does. I don’t think this has anything to do with Iterators.product itself, rather I guess comprehensions with AbstractArray ranges behave differently than comprehensions with arbitrary iterator ranges? Is this an intentional distinction?

2

I suspect the former is a bug. Perhaps report it as an issue?

Edit:

For what it’s worth, I can’t reproduce this issue. I get the expected 3-dimensional array in both cases.

julia> p = Iterators.product(1:2, 1:2)
Base.Iterators.Prod2{UnitRange{Int64},UnitRange{Int64}}(1:2, 1:2)

julia> [1 for x in p, y in 1:2]
2×2×2 Array{Int64,3}:
[:, :, 1] =
 1  1
 1  1

[:, :, 2] =
 1  1
 1  1

julia> [1 for x in collect(p), y in 1:2]
2×2×2 Array{Int64,3}:
[:, :, 1] =
 1  1
 1  1

[:, :, 2] =
 1  1
 1  1
3

FWIW, I get the same behavior as @rdeits, both on julia-0.5.1 and on latest master (b91b411). Also, Pkg.checkout(“Iterators”) makes no difference for me. @fengyang.wang, which julia version are you using?

4

Ah, the discreptancy is that you are using the Iterators package. I recommend that the Compat.Iterators (Base.Iterators in 0.6 and newer) package be used instead of Iterators. Iterators.product lacks shape information, whereas Compat.Iterators.product retains it.

Edit: To clarify, the difference is not between arrays and other iterables; the difference is between objects with shape information and objects without it. Iterators.jl was written, in large part, before the mechanisms for preserving shape information existed. In general, Base.Iterators (or Compat.Iterators for 0.5 support) is preferred.

2 Likes
5

Ah, I see, thanks! I had no idea that Base.Iterators existed in 0.6.

It still seems odd that shape-less iterators behave differently than 1-dimensional arrays in this case. Do you think it’s worth opening an issue about that?

6

I think this is intentional. It’s not possible to compute the shape ahead of time for length-less iterators like the Iterators.Product from Iterators.jl.