# Why is it that dataframe violates iterator interface?

**URL:** <https://discourse.julialang.org/t/why-is-it-that-dataframe-violates-iterator-interface/95906>\
**Category:** New to Julia\
**Tags:** dataframes, iterators\
**Created:** [March 11, 2023, 7:34am UTC](https://discourse.julialang.org/t/why-is-it-that-dataframe-violates-iterator-interface/95906 "2023-03-11T07:34:51Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![freeman](https://avatars.discourse-cdn.com/v4/letter/f/ec9cab/32.png) [@freeman](https://discourse.julialang.org/u/freeman)\
**Post date:** [March 11, 2023, 7:34am UTC](https://discourse.julialang.org/t/why-is-it-that-dataframe-violates-iterator-interface/95906/1 "2023-03-11T07:34:51Z")

</div>

```julia
using DataFrames
df = DataFrame(:a=>[1,2,3])
Base.IteratorSize(DataFrame) # Base.HasLength()
length(df) # method length(::DataFrame) doesn't exist

```

I’m ok with DataFrames not having `length`, but in that case why

```julia
Base.IteratorSize(DataFrame) == Base.HasLength()

```

---

<div class="post-metadata">

**Author:** ![bertschi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bertschi/32/33462_2.png) [@bertschi](https://discourse.julialang.org/u/bertschi)\
**Post date:** [March 11, 2023, 9:18am UTC](https://discourse.julialang.org/t/why-is-it-that-dataframe-violates-iterator-interface/95906/2 "2023-03-11T09:18:52Z")

</div>

To my understanding, `DataFrame` does not implement the iterator interface, i.e.,

```julia
julia> for x in df
           println(x)
       end
ERROR: AbstractDataFrame is not iterable. Use eachrow(df) to get a row iterator or eachcol(df) to get a column iterator

julia> Base.IteratorSize(typeof(eachrow(df)))
Base.HasShape{1}()

```

What you are seeing is just the default implementation of `IteratorSize`:

```julia
julia> @which Base.IteratorSize(DataFrame)
Base.IteratorSize(::Type) in Base at generator.jl:93

```

which is implemented as `IteratorSize(::Type) = HasLength() # HasLength is the default` and simply returns `HasLength()` for any type.

```julia

```

---

<div class="post-metadata">

**Author:** ![simsurace](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simsurace/32/30216_2.png) [@simsurace](https://discourse.julialang.org/u/simsurace)\
**Post date:** [March 11, 2023, 9:36am UTC](https://discourse.julialang.org/t/why-is-it-that-dataframe-violates-iterator-interface/95906/3 "2023-03-11T09:36:59Z")

</div>

But since `length` is not defined, maybe it would be better to overload this?

---

<div class="post-metadata">

**Author:** ![bertschi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bertschi/32/33462_2.png) [@bertschi](https://discourse.julialang.org/u/bertschi)\
**Post date:** [March 11, 2023, 11:21am UTC](https://discourse.julialang.org/t/why-is-it-that-dataframe-violates-iterator-interface/95906/4 "2023-03-11T11:21:24Z")

</div>

Maybe, but this would need to be done for any type then:

```julia
julia> struct MyType end

julia> length(MyType())
ERROR: MethodError: no method matching length(::MyType)

julia> Base.IteratorSize(MyType)
Base.HasLength()

```

Thus, currently `Base.IteratorSize` cannot be actually used to check that `length` will work as it’s only meaningful for types that actually care for the iterator interface. In particular, types not depending on that interface will not bother with overwriting this method (why should they).  
Arguably, the best fix would probably be to remove the default implementation of `IteratorSize`. Yet at this stage, this will probably be breaking … on the other hand, it would not matter much for code outside of iterators anyways (or why would you want to check `IteratorSize` on a type that is not iterable?).

---

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [March 11, 2023, 12:54pm UTC](https://discourse.julialang.org/t/why-is-it-that-dataframe-violates-iterator-interface/95906/5 "2023-03-11T12:54:56Z")

</div>

In 1.10 (iirc) we will get Tricks.jl style compile-time `hasmethod` we might be able to use that to give a implementation of `IteratorSize` that actually checks what methods are defined.  
Then you would only need to overload it is you were doing something odd.

We also could have it return an error status if the thing did not define `iterate` which iirc DataFrames do not. Right now there is no status for this since we assume people only try to use it on iterators, as was stated

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [March 11, 2023, 2:47pm UTC](https://discourse.julialang.org/t/why-is-it-that-dataframe-violates-iterator-interface/95906/6 "2023-03-11T14:47:01Z")

</div>

I do agree, calling `IteratorSize` on a completely generic object (i.e., one that you have no information if it is even an iterable) is a code smell. Your code should either assume it is (or is not) an iterable, or pass this information explicitly along the generic object.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [March 12, 2023, 7:43am UTC](https://discourse.julialang.org/t/why-is-it-that-dataframe-violates-iterator-interface/95906/7 "2023-03-12T07:43:47Z")

</div>

> [@oxinabox](#):
>
> if the thing did not define `iterate` which iirc DataFrames do not

They do, it’s the `iterate` method that throws:

```julia
julia> iterate(DataFrame())
ERROR: AbstractDataFrame is not iterable. Use eachrow(df) to get a row iterator or eachcol(df) to get a column iterator
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] iterate(#unused#::DataFrame)
   @ DataFrames ~/.julia/packages/DataFrames/LteEl/src/abstractdataframe/iteration.jl:23
 [3] top-level scope
   @ REPL[4]:1

```

---

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [March 13, 2023, 2:18am UTC](https://discourse.julialang.org/t/why-is-it-that-dataframe-violates-iterator-interface/95906/9 "2023-03-13T02:18:59Z")

</div>

this comment seems irrelevant to question…