# Loading first few lines from data file

**URL:** <https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412>\
**Category:** General Usage\
**Tags:** io, delimitedfiles\
**Created:** [March 3, 2019, 1:24am UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412 "2019-03-03T01:24:04Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![iamsuddhasattwa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iamsuddhasattwa/32/7441_2.png) [@iamsuddhasattwa](https://discourse.julialang.org/u/iamsuddhasattwa)\
**Post date:** [March 3, 2019, 1:24am UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/1 "2019-03-03T01:24:04Z")

</div>

Hi, I have a data file with 100 rows of data of the form  
1 2  
3 4  
5 6.1  
…

I want to load the first 10 lines as a 10X2 array. However, I do not see any option in readdlm() for doing that, although it should be an easy task.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [March 3, 2019, 8:08pm UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/2 "2019-03-03T20:08:53Z")

</div>

If the file is only 100 rows surely it won’t take more than a fraction of a second to read it, so wouldn’t you just do:

```julia
a = readdlm("my_100_row_file.txt")[1:10]

```

`readdlm` doesn’t seem to have an option to specify the number of rows to read, but you could easily do something yourself along the lines of:

```julia
function read_first_n(filename, n = 10)
  a = Array{Any}(undef, n)
  open(filename) do file
      for (i, ln) in enumerate(eachline(file))
        if i <= n
          a[i] = ln
        else 
          break
        end
      end
    end
    return a
end

```

or if you want to make your life easier just use the `CSV` package which is a lot more fully featured than the `DelimitedFiles` stdlib:

```julia
using CSV
CSV.read("my_100_row_file.txt", delim = ' ', rows = 10)

```

---

<div class="post-metadata">

**Author:** ![bennedich](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bennedich/32/4894_2.png) [@bennedich](https://discourse.julialang.org/u/bennedich)\
**Post date:** [March 3, 2019, 8:31pm UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/3 "2019-03-03T20:31:18Z")

</div>

I also don’t see how to do it with `readdlm`, but a combination of `Iterators.take` and `eachline` makes this quite easy to do yourself. In the example below, I read the first 3 lines only, and specify the type to be `Float64`:

```julia
julia> map.(s -> parse(Float64, s), split.(Iterators.take(eachline("test.txt"), 3), ' '))
3-element Array{Array{Float64,1},1}:
 [1.0, 2.0]
 [3.0, 4.0]
 [5.0, 6.1]

```

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [March 3, 2019, 8:45pm UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/4 "2019-03-03T20:45:13Z")

</div>

Since you can pass any IO handle to readdlm, you can do something like this:

```julia
open(`head -n10 $file`) do io
    readdlm(io)
end

```

Might be possible to just pass the command object directly to readdlm. I’m not at a computer so I haven’t tried any of this. As a philosophical matter, we try to avoid APIs with lots of options (head/tail/etc options to every function that reads a file) in favor of composable constructs (passing IO objects that can do some form of trucation to functions).

---

<div class="post-metadata">

**Author:** ![iamsuddhasattwa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iamsuddhasattwa/32/7441_2.png) [@iamsuddhasattwa](https://discourse.julialang.org/u/iamsuddhasattwa)\
**Post date:** [March 4, 2019, 1:07am UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/5 "2019-03-04T01:07:29Z")

</div>

Thank you for the reply. However, after having installed and loaded “Iterators”, I get the following error message. Perhaps there is an error in your code.

x = map.(s → parse(Float64, s), split.(Iterators.take(eachline(“Data.txt”), 3), ’ '))

ERROR: MethodError: no method matching size(::##16#18)  
Closest candidates are:  
size{N}(::Any, ::Integer, ::Integer, ::Integer…) at abstractarray.jl:48  
size(::BitArray{1}) at bitarray.jl:39  
size(::BitArray{1}, ::Any) at bitarray.jl:43  
…  
in broadcast\_shape(::Function, ::Base.Take{EachLine}, ::Char, ::Vararg{Char,N}) at ./broadcast.jl:31  
in broadcast\_t(::Function, ::Type{Any}, ::Function, ::Vararg{Any,N}) at ./broadcast.jl:213  
in broadcast(::Function, ::Function, ::Base.Take{EachLine}, ::Char) at ./broadcast.jl:230

---

<div class="post-metadata">

**Author:** ![iamsuddhasattwa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iamsuddhasattwa/32/7441_2.png) [@iamsuddhasattwa](https://discourse.julialang.org/u/iamsuddhasattwa)\
**Post date:** [March 4, 2019, 1:46am UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/6 "2019-03-04T01:46:07Z")

</div>

Thank you, this one actually worked. But when I said “10” in my posted question, I basically meant some arbitrary number. Is it possible to pass an argument N into the open() command ?

---

<div class="post-metadata">

**Author:** ![iamsuddhasattwa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iamsuddhasattwa/32/7441_2.png) [@iamsuddhasattwa](https://discourse.julialang.org/u/iamsuddhasattwa)\
**Post date:** [March 4, 2019, 1:49am UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/7 "2019-03-04T01:49:40Z")

</div>

Thank you for the reply. But there seems to be an error with the word undef in your second line. The following error message is created.

ERROR: UndefVarError: undef not defined

---

<div class="post-metadata">

**Author:** ![bicycle1885](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bicycle1885/32/107_2.png) [@bicycle1885](https://discourse.julialang.org/u/bicycle1885)\
**Post date:** [March 4, 2019, 3:26am UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/8 "2019-03-04T03:26:24Z")

</div>

Of couse, you can do. Try this way:

```julia
n = 10
file = "somefile.txt"
open(readlines, `head -n $(n) $(file)`)

```

---

<div class="post-metadata">

**Author:** ![bennedich](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bennedich/32/4894_2.png) [@bennedich](https://discourse.julialang.org/u/bennedich)\
**Post date:** [March 4, 2019, 5:06am UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/9 "2019-03-04T05:06:39Z")

</div>

> [@iamsuddhasattwa](#):
>
> Perhaps there is an error in your code.

No. I think what’s happening here is that you’re using Julia 0.6? If that’s the case, I’d strongly recommend pausing whatever development you’re doing and focusing on upgrading to Julia 1+.

I like the `readdlm` version, to avoid having to write the parsing logic yourself, I’m just a bit concerned about going through an external command that way. It doesn’t seem platform independent?

---

<div class="post-metadata">

**Author:** ![iamsuddhasattwa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iamsuddhasattwa/32/7441_2.png) [@iamsuddhasattwa](https://discourse.julialang.org/u/iamsuddhasattwa)\
**Post date:** [March 4, 2019, 5:15am UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/10 "2019-03-04T05:15:16Z")

</div>

My version is being given as v"0.5.1-pre+31", so I think it means 0.5.1. Is that too backdated ?  
Thank you for the tip !

---

<div class="post-metadata">

**Author:** ![bennedich](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bennedich/32/4894_2.png) [@bennedich](https://discourse.julialang.org/u/bennedich)\
**Post date:** [March 4, 2019, 5:20am UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/11 "2019-03-04T05:20:11Z")

</div>

Yes, that’s an old unmaintained version. We are on Julia 1.1 / 1.2 now. Lots of things have changed, which is why the examples given here don’t work for you. As I said, I think upgrading should be your number one priority.

---

<div class="post-metadata">

**Author:** ![iamsuddhasattwa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iamsuddhasattwa/32/7441_2.png) [@iamsuddhasattwa](https://discourse.julialang.org/u/iamsuddhasattwa)\
**Post date:** [March 4, 2019, 5:33am UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/12 "2019-03-04T05:33:29Z")

</div>

I will do that right away.

---

<div class="post-metadata">

**Author:** ![iamsuddhasattwa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iamsuddhasattwa/32/7441_2.png) [@iamsuddhasattwa](https://discourse.julialang.org/u/iamsuddhasattwa)\
**Post date:** [March 4, 2019, 5:34am UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/13 "2019-03-04T05:34:55Z")

</div>

Actually, the program gets into some kind of infinite loop when I do that

julia\> n = 10;

julia\> file = “Data.txt”;

julia\> x = open(readlines, `head -n$(n) $(file)')

Here, the execution keeps running.

---

<div class="post-metadata">

**Author:** ![iamsuddhasattwa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iamsuddhasattwa/32/7441_2.png) [@iamsuddhasattwa](https://discourse.julialang.org/u/iamsuddhasattwa)\
**Post date:** [March 15, 2019, 7:17pm UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/14 "2019-03-15T19:17:29Z")

</div>

> [@bicycle1885](#):
>
> ```julia
> open(readlines, `head -n $(n) $(file)`)
> 
> ```

Thanks it worked after I updated my Julia version.

---

<div class="post-metadata">

**Author:** ![iamsuddhasattwa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iamsuddhasattwa/32/7441_2.png) [@iamsuddhasattwa](https://discourse.julialang.org/u/iamsuddhasattwa)\
**Post date:** [March 15, 2019, 7:20pm UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/15 "2019-03-15T19:20:46Z")

</div>

> [@iamsuddhasattwa](#):
>
> Thanks it worked after I updated my Julia version.

Thanks it worked after I updated my Julia version. But the return type is Array{String,1}, not Array{Float64,2} as I had hoped.

---

<div class="post-metadata">

**Author:** ![bennedich](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bennedich/32/4894_2.png) [@bennedich](https://discourse.julialang.org/u/bennedich)\
**Post date:** [March 15, 2019, 8:02pm UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/16 "2019-03-15T20:02:23Z")

</div>

It’s because you’re using [`readlines`](https://docs.julialang.org/en/latest/base/io-network/#Base.readlines) which returns strings. As I stated above, personally I’m not a fan of mixing bash and Julia for things that can easily be done in just Julia, so I would do something like this:

```julia
julia> rows = 3;

julia> first_rows = Iterators.take(eachline("test.txt"), rows);

julia> data = reduce(vcat, map.(s -> parse(Float64, s), split.(first_rows, ' '))')
3×2 Array{Float64,2}:
 1.0 2.0
 3.0 4.0
 5.0 6.1

```

---

<div class="post-metadata">

**Author:** ![CameronBieganek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cameronbieganek/32/6915_2.png) [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)\
**Post date:** [March 30, 2020, 6:52pm UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/17 "2020-03-30T18:52:25Z")

</div>

> [@StefanKarpinski](#):
>
> As a philosophical matter, we try to avoid APIs with lots of options (head/tail/etc options to every function that reads a file) in favor of composable constructs (passing IO objects that can do some form of trucation to functions).

Given this philosophy, is there some way to create a file IO object that limits the number of lines that are read in? I mean, aside from calling the system `head` with `open(`head -n10 $file`)`. I can’t seem to find a way to create such an IO object, and thus it appears that the only way to limit the number of lines read by `readdlm` is to use `open(`head -n10 $file`)`.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [March 30, 2020, 7:34pm UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/18 "2020-03-30T19:34:32Z")

</div>

It seems like it doesn’t really need to be an IO object. For example:

```julia
julia> collect(Iterators.take(eachline("/usr/share/dict/words"), 10))
10-element Array{String,1}:
 "A"
 "a"
 "aa"
 "aal"
 "aalii"
 "aam"
 "Aani"
 "aardvark"
 "aardwolf"
 "Aaron"

```

---

<div class="post-metadata">

**Author:** ![Paul\_Soderlind](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paul_soderlind/32/1753_2.png) [@Paul\_Soderlind](https://discourse.julialang.org/u/Paul_Soderlind)\
**Post date:** [March 30, 2020, 8:33pm UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/19 "2020-03-30T20:33:21Z")

</div>

(on a lighter note)

Episode 2, season 3 of the British TV series Black Adder spends a lot of time on “aardvark” - number 8 in the list. (It evolves around S. Johnson’s dictionary and how he forgot about “aard…”) It’s (very) good fun.

---

<div class="post-metadata">

**Author:** ![CameronBieganek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cameronbieganek/32/6915_2.png) [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)\
**Post date:** [March 30, 2020, 10:24pm UTC](https://discourse.julialang.org/t/loading-first-few-lines-from-data-file/21412/20 "2020-03-30T22:24:47Z")

</div>

Ideally, I’d like to be able to take advantage of the row parsing that is built into `readdlm`, rather than having to read the rows in using `eachline` and parse them myself. It looks like `readdlm` can’t take in an iterator of strings representing rows:

```julia
julia> io() = IOBuffer("a\tb\tc\n1\t2\t3\n4\t5\t6\n7\t8\t9\n")
io (generic function with 1 method)

julia> readdlm(io())
4×3 Array{Any,2}:
  "a" "b" "c"
 1 2 3
 4 5 6
 7 8 9

julia> it = Iterators.take(eachline(io()), 2);

julia> readdlm(it)
ERROR: MethodError: no method matching readdlm_auto(::Base.Iterators.Take{Base.EachLine{Base.GenericIOBuffer{Array{UInt8,1}}}}, ::Char, ::Type{Float64}, ::Char, ::Bool)

```

It is theoretically possible to accomplish by using this horrible unsightly hack:

```julia
julia> readdlm(IOBuffer(join(Iterators.take(eachline(io()), 2), '\n')))
2×3 Array{Any,2}:
  "a" "b" "c"
 1 2 3

```

But it would be much nicer if I could just do this:

```julia
readdlm(file; limit=2)

```

This would be useful when you have a lot of large files and you just want to read in the first few lines for each of them. I know that this can be done with CSV.jl, but it seems like `readdlm` ought to support this also.