# Iterating n items at a time and filling

**URL:** <https://discourse.julialang.org/t/iterating-n-items-at-a-time-and-filling/47045>\
**Category:** General Usage\
**Created:** [September 22, 2020, 12:36am UTC](https://discourse.julialang.org/t/iterating-n-items-at-a-time-and-filling/47045 "2020-09-22T00:36:59Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [September 22, 2020, 12:36am UTC](https://discourse.julialang.org/t/iterating-n-items-at-a-time-and-filling/47045/1 "2020-09-22T00:36:59Z")

</div>

In Slack, @mastrof wanted the equivalent of the following Python 3 code which returns `n` items at a time and pads with `fillvalue` if there are less than `n` items remaining

```python
def f(iterable, n, fillvalue=None):
    args = [iter(iterable)] * n
    return zip_longest(*args, fillvalue=fillvalue) 

```

> **[Slack is your productivity platform](https://julialang.slack.com/?redir=%2Farchives%2FC6A044SQH%2Fp1600715804339800)**
>
> Slack is a new way to communicate with your team. It’s faster, better organized, and more secure than email.

We currently do not have `zip_longest` in IterTools.jl although there is a PR:

> <https://github.com/JuliaCollections/IterTools.jl/pull/80>
>
> This is an implementation of #57.

Additionally, the iteration protocol is distinct in Julia from Python.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [September 22, 2020, 12:55am UTC](https://discourse.julialang.org/t/iterating-n-items-at-a-time-and-filling/47045/2 "2020-09-22T00:55:43Z")

</div>

After @jakobnissen recommended `Iterators.partition`, I proposed the following solution:

```julia
function ntaker_gen(iterable, n, fillvalue=nothing)
   padded_iterable = Iterators.flatten( (iterable, Iterators.repeated(fillvalue,n-1)) )
   n_partition = Iterators.partition( padded_iterable, n )
   (x for x in n_partition if length(x) == n)
end

ntaker(args...) = collect( ntaker_gen(args...) )

```

This results in the following:

```julia
julia> ntaker(1:6, 3, 0)
2-element Array{Array{Any,1},1}:
 [1, 2, 3]
 [4, 5, 6]

julia> ntaker(1:7, 3, 0)
3-element Array{Array{Any,1},1}:
 [1, 2, 3]
 [4, 5, 6]
 [7, 0, 0]

julia> ntaker(1:8, 3, 0)
3-element Array{Array{Any,1},1}:
 [1, 2, 3]
 [4, 5, 6]
 [7, 8, 0]

julia> ntaker(1:9, 3, 0)
3-element Array{Array{Any,1},1}:
 [1, 2, 3]
 [4, 5, 6]
 [7, 8, 9]

```

Is there a better solution?

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [September 22, 2020, 1:03am UTC](https://discourse.julialang.org/t/iterating-n-items-at-a-time-and-filling/47045/3 "2020-09-22T01:03:06Z")

</div>

Here is a one line version:

```julia
julia> function ntaker_one_liner(iterable, n, fillvalue=nothing)
           [[x; fill(fillvalue, n-length(x))] for x in Iterators.partition( iterable, n ) ]
       end
ntaker_one_liner (generic function with 2 methods)

julia> ntaker_one_liner(1:6, 3, 0)
2-element Array{Array{Int64,1},1}:
 [1, 2, 3]
 [4, 5, 6]

julia> ntaker_one_liner(1:7, 3, 0)
3-element Array{Array{Int64,1},1}:
 [1, 2, 3]
 [4, 5, 6]
 [7, 0, 0]

julia> ntaker_one_liner(1:8, 3, 0)
3-element Array{Array{Int64,1},1}:
 [1, 2, 3]
 [4, 5, 6]
 [7, 8, 0]

julia> ntaker_one_liner(1:9, 3, 0)
3-element Array{Array{Int64,1},1}:
 [1, 2, 3]
 [4, 5, 6]
 [7, 8, 9]

```

---

<div class="post-metadata">

**Author:** ![CameronBieganek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cameronbieganek/32/6915_2.png) [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)\
**Post date:** [September 22, 2020, 3:51am UTC](https://discourse.julialang.org/t/iterating-n-items-at-a-time-and-filling/47045/4 "2020-09-22T03:51:23Z")

</div>

```julia
using Base.Iterators

function foo(itr, n, fillvalue)
    ntake = ceil(Int, length(itr)/n)
    extended_itr = flatten( ( itr, repeated(fillvalue) ) )
    take(partition(extended_itr, n), ntake)
end

```

However, note that for some reason this produces a type instability. Maybe the compiler is coughing up on all those nested iterators?

```julia
julia> collect(foo(1:7, 3, 0))
3-element Array{Array{Any,1},1}:
 [1, 2, 3]
 [4, 5, 6]
 [7, 0, 0]

julia> collect(foo(1:8, 3, 0))
3-element Array{Array{Any,1},1}:
 [1, 2, 3]
 [4, 5, 6]
 [7, 8, 0]

julia> collect(foo(1:9, 3, 0))
3-element Array{Array{Any,1},1}:
 [1, 2, 3]
 [4, 5, 6]
 [7, 8, 9]

julia> collect(foo(1:10, 3, 0))
4-element Array{Array{Any,1},1}:
 [1, 2, 3] 
 [4, 5, 6] 
 [7, 8, 9] 
 [10, 0, 0]

```

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [September 22, 2020, 4:48am UTC](https://discourse.julialang.org/t/iterating-n-items-at-a-time-and-filling/47045/5 "2020-09-22T04:48:30Z")

</div>

Thanks for this version. I understand how to use `take` much better now.

One assumption this makes is that `length(itr)` exists but `length` is only an optional part of the iteration interface. For example filtered iterators may not have a known length _a priori_:

```julia
julia> filtered_iter = (x for x in 1:7 if mod(x,2))
Base.Generator{Base.Iterators.Filter{var"#5#6",UnitRange{Int64}},typeof(identity)}(identity, Base.Iterators.Filter{var"#5#6",UnitRange{Int64}}(var"#5#6"(), 1:7))

julia> length(filtered_iter)
ERROR: MethodError: no method matching length(::Base.Iterators.Filter{var"#5#6",UnitRange{Int64}})
Closest candidates are:
  length(::Cmd) at process.jl:639
  length(::Base.Iterators.Flatten{Tuple{}}) at iterators.jl:1061
  length(::BitSet) at bitset.jl:365
  ...
Stacktrace:
 [1] length(::Base.Generator{Base.Iterators.Filter{var"#5#6",UnitRange{Int64}},typeof(identity)}) at ./generator.jl:50
 [2] top-level scope at REPL[5]:1

julia> function foo(itr, n, fillvalue)
           ntake = ceil(Int, length(itr)/n)
           extended_itr = flatten( ( itr, repeated(fillvalue) ) )
           take(partition(extended_itr, n), ntake)
       end
foo (generic function with 1 method)

julia> foo(filtered_iter, 3, 0)
ERROR: MethodError: no method matching length(::Base.Iterators.Filter{var"#5#6",UnitRange{Int64}})
Closest candidates are:
  length(::Cmd) at process.jl:639
  length(::Base.Iterators.Flatten{Tuple{}}) at iterators.jl:1061
  length(::BitSet) at bitset.jl:365
  ...
Stacktrace:
 [1] length(::Base.Generator{Base.Iterators.Filter{var"#5#6",UnitRange{Int64}},typeof(identity)}) at ./generator.jl:50
 [2] foo(::Base.Generator{Base.Iterators.Filter{var"#5#6",UnitRange{Int64}},typeof(identity)}, ::Int64, ::Int64) at ./REPL[7]:2
 [3] top-level scope at REPL[8]:1

```

[https://docs.julialang.org/en/v1/manual/interfaces/#man-interface-iteration](https://docs.julialang.org/en/v1/manual/interfaces/#man-interface-iteration)

---

<div class="post-metadata">

**Author:** ![mastrof](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mastrof/32/17443_2.png) [@mastrof](https://discourse.julialang.org/u/mastrof)\
**Post date:** [September 22, 2020, 7:46am UTC](https://discourse.julialang.org/t/iterating-n-items-at-a-time-and-filling/47045/6 "2020-09-22T07:46:33Z")

</div>

Thank you @mkitti for moving this to Discourse.  
I originally asked on Slack thinking that the Python code could be trivially translated to Julia and wouldn’t be worth a topic here. Looks like I was very wrong 🙂