# Importing only specific lines from large csv file

**URL:** <https://discourse.julialang.org/t/importing-only-specific-lines-from-large-csv-file/108672>\
**Category:** Data\
**Tags:** csv\
**Created:** [January 11, 2024, 10:59am UTC](https://discourse.julialang.org/t/importing-only-specific-lines-from-large-csv-file/108672 "2024-01-11T10:59:44Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![aris](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@aris](https://discourse.julialang.org/u/aris)\
**Post date:** [January 11, 2024, 10:59am UTC](https://discourse.julialang.org/t/importing-only-specific-lines-from-large-csv-file/108672/1 "2024-01-11T10:59:44Z")

</div>

Greetings everyone,

I am dealing with a very large CSV file, and I would like to only read specific parts of it and import them as a matrix.  
Effectively, I want something like:

```julia
using DelimitedFiles
data = readdlm("myfile.csv")[n:m,:]

```

where “n” is the 1st row and “m” is the last row I’m interested in.  
If I understand correcly, the code above loads the whole thing in the memory, which is not good if your data file is a few million rows long.  
This is also going to be part of a loop, where in each iteration I load a different chunk from the csv, so performance matters.

What would be the most efficient and simple way to do this?

---

<div class="post-metadata">

**Author:** ![yakir12](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yakir12/32/297_2.png) [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Post date:** [January 11, 2024, 11:17am UTC](https://discourse.julialang.org/t/importing-only-specific-lines-from-large-csv-file/108672/2 "2024-01-11T11:17:14Z")

</div>

Checkout these examples in CSV.jl’s documentation:

> [@aris](#):
>
> where “n” is the 1st row

> **[Examples · CSV.jl](https://csv.juliadata.org/stable/examples.html#skipto_example)**
>
> Documentation for CSV.jl.

> [@aris](#):
>
> “m” is the last row

> **[Examples · CSV.jl](https://csv.juliadata.org/stable/examples.html#footerskip_example)**
>
> Documentation for CSV.jl.

---

<div class="post-metadata">

**Author:** ![hellemo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hellemo/32/324_2.png) [@hellemo](https://discourse.julialang.org/u/hellemo)\
**Post date:** [January 11, 2024, 11:21am UTC](https://discourse.julialang.org/t/importing-only-specific-lines-from-large-csv-file/108672/3 "2024-01-11T11:21:47Z")

</div>

`CSV.jl` has functionality for reading large CSV files in chunks already: [Reading · CSV.jl](https://csv.juliadata.org/stable/reading.html#CSV.Chunks)

To do exactly what you ask, you could e.g. do something like this with `skipto` and `limit`, but using the chunk iterator might be a better solution.

```julia-repl
julia> using CSV

julia> data = """
       a,b,c
       1,2,3
       4,5,6
       7,8,9
       totals: 12, 15, 18
       grand total: 45
       """
"a,b,c\n1,2,3\n4,5,6\n7,8,9\ntotals: 12, 15, 18\ngrand total: 45\n"

julia> file = CSV.File(IOBuffer(data);skipto=3, limit=2)
2-element CSV.File:
 CSV.Row: (var"a,b,c" = String7("4,5,6"),)
 CSV.Row: (var"a,b,c" = String7("7,8,9"),)

```

---

<div class="post-metadata">

**Author:** ![aris](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@aris](https://discourse.julialang.org/u/aris)\
**Post date:** [January 11, 2024, 11:26am UTC](https://discourse.julialang.org/t/importing-only-specific-lines-from-large-csv-file/108672/5 "2024-01-11T11:26:03Z")

</div>

Thanks for the reply!

---

<div class="post-metadata">

**Author:** ![aris](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@aris](https://discourse.julialang.org/u/aris)\
**Post date:** [January 11, 2024, 11:37am UTC](https://discourse.julialang.org/t/importing-only-specific-lines-from-large-csv-file/108672/6 "2024-01-11T11:37:13Z")

</div>

Thank you for your reply, hellemo.  
It works well, but it imports the data as a CSV.file format.  
Is there a way to use a matrix array instead?

---

<div class="post-metadata">

**Author:** ![hellemo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hellemo/32/324_2.png) [@hellemo](https://discourse.julialang.org/u/hellemo)\
**Post date:** [January 11, 2024, 11:59am UTC](https://discourse.julialang.org/t/importing-only-specific-lines-from-large-csv-file/108672/7 "2024-01-11T11:59:14Z")

</div>

You can use `stack` for example:

```julia-repl
julia> stack(CSV.File(IOBuffer(data);delim=',',skipto=3, limit=2);dims=1)
2×3 Matrix{Any}:
 4 5 6
 7 8 9

```

But here we don’t get the type inferred, one solution (maybe not most efficient) would be to use `identity` to narrow the type in case they are all the same (as here):

```julia-repl
julia> identity.(stack(CSV.File(IOBuffer(data);delim=',',skipto=3, limit=2);dims=1))
2×3 Matrix{Int64}:
 4 5 6
 7 8 9

```

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [January 11, 2024, 12:23pm UTC](https://discourse.julialang.org/t/importing-only-specific-lines-from-large-csv-file/108672/8 "2024-01-11T12:23:05Z")

</div>

It is worth noting that reading into a dataframe infers the types, and from there arrays can be used.