# Iterator yielding tuples vs arrays

**URL:** <https://discourse.julialang.org/t/iterator-yielding-tuples-vs-arrays/114686>\
**Category:** General Usage\
**Created:** [May 24, 2024, 3:35pm UTC](https://discourse.julialang.org/t/iterator-yielding-tuples-vs-arrays/114686 "2024-05-24T15:35:30Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Marco-Congedo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marco-congedo/32/7321_2.png) [@Marco-Congedo](https://discourse.julialang.org/u/Marco-Congedo)\
**Post date:** [May 24, 2024, 3:35pm UTC](https://discourse.julialang.org/t/iterator-yielding-tuples-vs-arrays/114686/1 "2024-05-24T15:35:30Z")

</div>

This iterator yields

```julia
    n=3
    P=Iterators.product(((1, -1) for i=1:n)...)
    foreach(println, P)

(1, 1, 1)
(-1, 1, 1)
(1, -1, 1)
(-1, -1, 1)
(1, 1, -1)
(-1, 1, -1)
(1, -1, -1)
(-1, -1, -1)

```

which is what i need for any `n`.  
How should i modify the iterator to yield arrays instead of tuples, that is:

```julia
 [1, 1, 1]
[-1, 1, 1]
[1, -1, 1]
[-1, -1, 1]
[1, 1, -1]
[-1, 1, -1]
[1, -1, -1]
[-1, -1, -1]

```

?

---

<div class="post-metadata">

**Author:** ![bertschi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bertschi/32/33462_2.png) [@bertschi](https://discourse.julialang.org/u/bertschi)\
**Post date:** [May 24, 2024, 3:54pm UTC](https://discourse.julialang.org/t/iterator-yielding-tuples-vs-arrays/114686/2 "2024-05-24T15:54:47Z")

</div>

```julia
foreach(println, map(collect, P))
# or
foreach(println, collect.(P))

```

---

<div class="post-metadata">

**Author:** ![Marco-Congedo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marco-congedo/32/7321_2.png) [@Marco-Congedo](https://discourse.julialang.org/u/Marco-Congedo)\
**Post date:** [May 24, 2024, 4:04pm UTC](https://discourse.julialang.org/t/iterator-yielding-tuples-vs-arrays/114686/3 "2024-05-24T16:04:12Z")

</div>

I don’t want to collect the iterations, i would like that the iterator format the iterations as array, that is, that each element of the iterator behaves like an array, not as a tuple. This is for example what `permutations` in Combinatorics.jl does.  
For example:

```julia
using Combinatorics
Q=permutations(1:3)
foreach(println, Q)

[1, 2, 3]
[1, 3, 2]
[2, 1, 3]
[2, 3, 1]
[3, 1, 2]
[3, 2, 1]

```

---

<div class="post-metadata">

**Author:** ![Jeff\_Emanuel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeff_emanuel/32/15440_2.png) [@Jeff\_Emanuel](https://discourse.julialang.org/u/Jeff_Emanuel)\
**Post date:** [May 24, 2024, 4:16pm UTC](https://discourse.julialang.org/t/iterator-yielding-tuples-vs-arrays/114686/4 "2024-05-24T16:16:17Z")

</div>

How is this?

```julia
Q=Iterators.map(collect, P)
foreach(println, Q)
[1, 1, 1]
[-1, 1, 1]
[1, -1, 1]
[-1, -1, 1]
[1, 1, -1]
[-1, 1, -1]
[1, -1, -1]
[-1, -1, -1]

```

[/quote]

If don’t like that, then I think you need to reimplement `Iterators.Product`.

---

<div class="post-metadata">

**Author:** ![Marco-Congedo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marco-congedo/32/7321_2.png) [@Marco-Congedo](https://discourse.julialang.org/u/Marco-Congedo)\
**Post date:** [May 24, 2024, 4:18pm UTC](https://discourse.julialang.org/t/iterator-yielding-tuples-vs-arrays/114686/5 "2024-05-24T16:18:12Z")

</div>

> [@Marco-Congedo](#):
>
> `Q=Iterators.map(collect, P)`

I guess this would still collect the iterations, wouldn’t it? I want a lazy iterator, very fast to create for large n without collecting anything.

---

<div class="post-metadata">

**Author:** ![Jeff\_Emanuel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeff_emanuel/32/15440_2.png) [@Jeff\_Emanuel](https://discourse.julialang.org/u/Jeff_Emanuel)\
**Post date:** [May 24, 2024, 4:20pm UTC](https://discourse.julialang.org/t/iterator-yielding-tuples-vs-arrays/114686/6 "2024-05-24T16:20:03Z")

</div>

FWIW

[https://docs.julialang.org/en/v1/base/iterators/#Base.Iterators.map](https://docs.julialang.org/en/v1/base/iterators/#Base.Iterators.map)

> ```julia
> Iterators.map(f, iterators...)
> 
> ```
> 
> Create a _lazy_ mapping.

---

<div class="post-metadata">

**Author:** ![Marco-Congedo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marco-congedo/32/7321_2.png) [@Marco-Congedo](https://discourse.julialang.org/u/Marco-Congedo)\
**Post date:** [May 24, 2024, 4:21pm UTC](https://discourse.julialang.org/t/iterator-yielding-tuples-vs-arrays/114686/7 "2024-05-24T16:21:27Z")

</div>

Great, thanks @Jeff_Emanuel. It does exactly what i need

---

<div class="post-metadata">

**Author:** ![bertschi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bertschi/32/33462_2.png) [@bertschi](https://discourse.julialang.org/u/bertschi)\
**Post date:** [May 24, 2024, 4:49pm UTC](https://discourse.julialang.org/t/iterator-yielding-tuples-vs-arrays/114686/8 "2024-05-24T16:49:53Z")

</div>

Right, tend to forget that `map` (eager) and `Iterators.map` (lazy) are not the same. In any case, here is a lazy alternative using [Transducers.jl](https://github.com/JuliaFolds/Transducers.jl):

```julia
P = Iterators.product(((1, -1) for i=1:n)...) |> Transducers.Map(collect)

```

---

<div class="post-metadata">

**Author:** ![Marco-Congedo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marco-congedo/32/7321_2.png) [@Marco-Congedo](https://discourse.julialang.org/u/Marco-Congedo)\
**Post date:** [May 24, 2024, 5:25pm UTC](https://discourse.julialang.org/t/iterator-yielding-tuples-vs-arrays/114686/9 "2024-05-24T17:25:36Z")

</div>

> [@bertschi](#):
>
> `P = Iterators.product(((1, -1) for i=1:n)...) |> Transducers.Map(collect)`

The think is, the usage of the iterator gets slower. This is true also for the solution given by @Jeff_Emanuel :

```julia
    using BenchmarkTools, Transducers

    n=15
    y=randn(n)

    P=Iterators.map(collect, Iterators.product(((1., -1.) for i=1:n)...)) # yields arrays
    function foo1(P, y)
        for p in P
            a=p.*y
        end
    end

    Q=Iterators.product(((1., -1.) for i=1:n)...) # yields tuples
    function foo2(Q, y)
        for q in Q
            a=q.*y
        end
    end

    R=Iterators.product(((1., -1.) for i=1:n)...) |> Transducers.Map(collect) 
    function foo3(R, y)
        for r in R
            a=r.*y
        end
    end

    @btime foo1($P, $y) # 2.486 ms (65536 allocations: 11.00 MiB)
    @btime foo2($Q, $y) # 1.150 ms (32768 allocations: 5.50 MiB)
    @btime foo3($R, $y) # 4.689 ms (98307 allocations: 20.00 MiB)

```

---

<div class="post-metadata">

**Author:** ![Jeff\_Emanuel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeff_emanuel/32/15440_2.png) [@Jeff\_Emanuel](https://discourse.julialang.org/u/Jeff_Emanuel)\
**Post date:** [May 24, 2024, 5:39pm UTC](https://discourse.julialang.org/t/iterator-yielding-tuples-vs-arrays/114686/10 "2024-05-24T17:39:24Z")

</div>

`product` generates tuples. If you must convert the tuples to arrays, then you have to pay for their allocation. To avoid that conversion you need to implement your own `product` to generate arrays instead.

[https://docs.julialang.org/en/v1/manual/interfaces/#man-interface-iteration](https://docs.julialang.org/en/v1/manual/interfaces/#man-interface-iteration)

> <https://github.com/JuliaLang/julia/blob/0b4590a5507d3f3046e5bafc007cacbbfc9b310b/base/iterators.jl#L999>

---

<div class="post-metadata">

**Author:** ![mikmoore](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikmoore/32/31109_2.png) [@mikmoore](https://discourse.julialang.org/u/mikmoore)\
**Post date:** [May 24, 2024, 5:51pm UTC](https://discourse.julialang.org/t/iterator-yielding-tuples-vs-arrays/114686/11 "2024-05-24T17:51:22Z")

</div>

> [@Jeff\_Emanuel](#):
>
> To avoid that conversion you need to implement your own `product` to generate arrays instead.

An alternative is to use the existing `StaticArrays.jl` package to wrap the tuples. A `StaticArray` is just a `Tuple` wrapped to behave like an array. It incurs no heap allocations in most situations.

```julia-repl
julia> using StaticArrays

julia> Iterators.map(SVector, P) |> collect # remove the collect in actual use
2×2×2 Array{SVector{3, Int64}, 3}:
[:, :, 1] =
 [1, 1, 1] [1, -1, 1]
 [-1, 1, 1] [-1, -1, 1]

[:, :, 2] =
 [1, 1, -1] [1, -1, -1]
 [-1, 1, -1] [-1, -1, -1]

```

---

<div class="post-metadata">

**Author:** ![Marco-Congedo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marco-congedo/32/7321_2.png) [@Marco-Congedo](https://discourse.julialang.org/u/Marco-Congedo)\
**Post date:** [May 24, 2024, 6:17pm UTC](https://discourse.julialang.org/t/iterator-yielding-tuples-vs-arrays/114686/12 "2024-05-24T18:17:07Z")

</div>

> [@mikmoore](#):
>
> `Iterators.map(SVector, P)`

That’s a nice trick. It seems like I get the same performance as of the original iterator.

```julia
    using BenchmarkTools, Transducers, StaticArrays

    n=15
    y=randn(n)

    P=Iterators.map(collect, Iterators.product(((1., -1.) for i=1:n)...)) # yields arrays
    function foo1(P, y)
        for p in P
            a=p.*y
        end
    end

    Q=Iterators.product(((1., -1.) for i=1:n)...) # yields typles
    function foo2(Q, y)
        for q in Q
            a=q.*y
        end
    end

    R=Iterators.product(((1., -1.) for i=1:n)...) |> Transducers.Map(collect) # yields arrays
    function foo3(R, y)
        for r in R
            a=r.*y
        end
    end

    S=Iterators.map(SVector, Q)
    function foo4(S, y)
        for s in S
            a=s.*y
        end
    end

    @btime foo1($P, $y) # 2.486 ms (65536 allocations: 11.00 MiB)
    @btime foo2($Q, $y) # 1.150 ms (32768 allocations: 5.50 MiB)
    @btime foo3($R, $y) # 4.689 ms (98307 allocations: 20.00 MiB)
    @btime foo4($S, $y) # 1.137 ms (32768 allocations: 5.50 MiB)

    @btime S=Iterators.map(SVector, Q) # 371.707 ns (2 allocations: 512 bytes)

```