# How to use CSVFile as iterable?

**URL:** <https://discourse.julialang.org/t/how-to-use-csvfile-as-iterable/28005>\
**Category:** New to Julia\
**Created:** [August 26, 2019, 6:19pm UTC](https://discourse.julialang.org/t/how-to-use-csvfile-as-iterable/28005 "2019-08-26T18:19:36Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![gabomgp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gabomgp/32/1918_2.png) [@gabomgp](https://discourse.julialang.org/u/gabomgp)\
**Post date:** [August 26, 2019, 6:19pm UTC](https://discourse.julialang.org/t/how-to-use-csvfile-as-iterable/28005/1 "2019-08-26T18:19:36Z")

</div>

Hello again. I’m testing reading a large CSV using CSVFiles instead of CSV.jl because the CSV file [is too large for CSV.jl on Windows](https://discourse.julialang.org/t/is-it-possible-to-iterate-over-a-very-large-csv-in-windows/27827). But I can not use the CSVFile as a iterable. I’m trying:

```julia
function mylength(iter)
    n=0
        for i in iter  
            n+=1
        end
    return n
end

function test(src:: String)
    table = load(File(format"CSV", src);
        colnames=[:day, :glnprovider, :glnretailerlocation, :gtin, :inventory, :cost, :sales, :price],
        colparsers=[Date, UInt64, UInt64, UInt64, Float32, Float32, Float32, Float32],
        header_exists=false
    )
    return table |> mylength
end

test("D:\\Data\\03012019_03312019_17440.csv.gz")

```

I’m hoping to use the _table_ as iterable to calculate diferent things in streaming. But, i’m getting the error:

```
MethodError: no method matching iterate(::CSVFiles.CSVFile)

```

So, i’m found the document [developer guide](https://github.com/queryverse/IterableTables.jl/blob/master/docs/src/developerguide.md) from iterable tables. In that document, the author says that we can call the method `getiterator`. But when i tried that, the error is:

```
UndefVarError: getiterator not defined

```

So, how can i use a iterable table (the CSV file) as a iterator? getting the iterator somehow?

---

<div class="post-metadata">

**Author:** ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Post date:** [August 26, 2019, 6:32pm UTC](https://discourse.julialang.org/t/how-to-use-csvfile-as-iterable/28005/2 "2019-08-26T18:32:55Z")

</div>

`getiterator` is defined in [IteratorInterfaceExtensions.jl](https://github.com/queryverse/IteratorInterfaceExtensions.jl), so you need to load that package.

But be warned: [CSVFiles.jl](https://github.com/queryverse/CSVFiles.jl) currently reads everything into memory, and then iterates from that. So if `load("foo.csv") |> DataFrame` doesn’t work because of memory limitations, then using `getiterator` will probably also not work (still worth a try, of course).

If you don’t need all of the columns of the file, you can try the new skip column feature that I’ve added to `TextParse#master`: make sure you are using that (`pkg> add TextParse#master`), and then something like `load("foo.csv", colparsers=Dict(:colA=>nothing, :colC=>nothing)) |> DataFrame` should work. In that case, `colA` and `colB` are not being loaded into memory at all.

My next project is to integrate the skip column feature with [Query.jl](https://github.com/queryverse/Query.jl), so that something like `load("foo.csv") |> @select(-:colA, -:colC) |> DataFrame` automatically skips those columns during load. No promise on timing, though 🙂

EDIT: Oh, and I also plan to add a fully streaming mode at some point. Almost all the pieces for that exist already, so it actually shouldn’t be too difficult, but again, no timeline right now.

---

<div class="post-metadata">

**Author:** ![gabomgp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gabomgp/32/1918_2.png) [@gabomgp](https://discourse.julialang.org/u/gabomgp)\
**Post date:** [August 26, 2019, 9:21pm UTC](https://discourse.julialang.org/t/how-to-use-csvfile-as-iterable/28005/3 "2019-08-26T21:21:15Z")

</div>

Thanks for your time. I’ll check just in case.

---

<div class="post-metadata">

**Author:** ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Post date:** [August 26, 2019, 10:25pm UTC](https://discourse.julialang.org/t/how-to-use-csvfile-as-iterable/28005/4 "2019-08-26T22:25:53Z")

</div>

Great, any feedback on whether it works would be most welcome 🙂