# Read vector from data file

**URL:** https://discourse.julialang.org/t/read-vector-from-data-file/108945
**Category:** Data
**Tags:** csv, io
**Created:** [January 18, 2024, 9:46am UTC](https://discourse.julialang.org/t/read-vector-from-data-file/108945 "2024-01-18T09:46:21Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![Matt\_jl](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/matt_jl/32/52364_2.png) [@Matt\_jl](https://discourse.julialang.org/u/Matt_jl)
#### Post date: [January 18, 2024, 9:46am UTC](https://discourse.julialang.org/t/read-vector-from-data-file/108945/1 "2024-01-18T09:46:21Z")

</div>

Hi  
I’m currently trying to read a vector from a .dat file. the structure of the file is not complicated and I currently managed to read the rows/columns that I need by using the following code:

```julia
using DelimitedFiles
function get_data(path_to_file::String)
    data = readdlm(path_to_file)
    z = data[1145:1183, 1] # first col
    pdz = data[1145:1183, 2] # Second col
    return z, pdz
end

```

as you can see I need only some specific rows (contained by two text separators)

the file is structured as follows:

```julia
#... data percentile .....
0 1 2 3 4 5 6
(I don't need them, 1 row multiple columns)

# ... data 1...
    0.1000 1.825E-029
    0.3000 6.247E-016
    0.5000 3.227E-007
    0.7000 4.726E-008
    0.9000 3.678E-008
... (data that I need, multiple rows, 2 col)
...
#.... data 2 ....
(I don't need them)

... and so on

```

this work but I found my solution quite inelegant.  
for reference in Python, using numpy this can be obtained with just 1 line of code:

```julia
z_p, pdz_p=np.genfromtxt(path_to_file,unpack=True,skip_header=1144,max_rows=40)

```

Is there any solution out there similar to python/numpy?  
I haven’t benchmarked the codes but I bet my current implementation Is slower to numpy.

---

<div class="post-metadata">

### Author: ![TheLateKronos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thelatekronos/32/12824_2.png) [@TheLateKronos](https://discourse.julialang.org/u/TheLateKronos)
#### Post date: [January 18, 2024, 10:22am UTC](https://discourse.julialang.org/t/read-vector-from-data-file/108945/2 "2024-01-18T10:22:28Z")

</div>

Have you tried CSV.jl? I believe it should be able to do what you want.

As a sidenote, it might be easier to clean up your data file and then read a nicely formatted file, than directly reading a file with messy formatting. It might require a temp-file, but given that you know the start row it seems like you only want to read a single file, so then that is no problem.

Also, do not worry about performance unless you have to. Premature optimization can take a lot of time and make the code less readable and/or less general. If the difference is 1 vs 5 seconds, as a one-time cost (or 1 vs 5 milliseconds more realistically), then there is little to actually be gained.

---

<div class="post-metadata">

### Author: ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)
#### Post date: [January 18, 2024, 10:57am UTC](https://discourse.julialang.org/t/read-vector-from-data-file/108945/3 "2024-01-18T10:57:17Z")

</div>

The easiest solution is perhaps [given here](https://discourse.julialang.org/t/importing-only-specific-lines-from-large-csv-file/108672/3).

---

<div class="post-metadata">

### Author: ![Matt\_jl](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/matt_jl/32/52364_2.png) [@Matt\_jl](https://discourse.julialang.org/u/Matt_jl)
#### Post date: [January 18, 2024, 10:57am UTC](https://discourse.julialang.org/t/read-vector-from-data-file/108945/4 "2024-01-18T10:57:33Z")

</div>

I’m looking with CSV.File but it doesn’t work as I expect,  
I don’t know if I can unpack the results and for some reason it tries to generate the columns based on the first uncommented line of the file, not the ones I’m reading.

```julia
data = CSV.File(path_to_file, skipto=1144, limit=40, header=1144,ignorerepeated=true, comment="#")

```

---

<div class="post-metadata">

### Author: ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)
#### Post date: [January 18, 2024, 11:26am UTC](https://discourse.julialang.org/t/read-vector-from-data-file/108945/5 "2024-01-18T11:26:09Z")

</div>

Your MWE doesn’t seem to show a header. Try `header=false`

---

<div class="post-metadata">

### Author: ![Matt\_jl](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/matt_jl/32/52364_2.png) [@Matt\_jl](https://discourse.julialang.org/u/Matt_jl)
#### Post date: [January 18, 2024, 1:19pm UTC](https://discourse.julialang.org/t/read-vector-from-data-file/108945/6 "2024-01-18T13:19:57Z")

</div>

nope.  
for some reasons it reads 19 columns, 2 of which are the 2 that I want but I dont’ get where is it getting the remaining columns

---

<div class="post-metadata">

### Author: ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)
#### Post date: [January 18, 2024, 1:22pm UTC](https://discourse.julialang.org/t/read-vector-from-data-file/108945/7 "2024-01-18T13:22:56Z")

</div>

Have you tried: `delim=' '` ?

---

<div class="post-metadata">

### Author: ![Matt\_jl](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/matt_jl/32/52364_2.png) [@Matt\_jl](https://discourse.julialang.org/u/Matt_jl)
#### Post date: [January 18, 2024, 1:26pm UTC](https://discourse.julialang.org/t/read-vector-from-data-file/108945/8 "2024-01-18T13:26:52Z")

</div>

ok it works!  
it was my error, i thought that the key ‘ignorerepeated = true’ was enough but I was wrong.  
Thank you!  
the final code:

```julia
data = CSV.File(data_file; skipto=1144,limit=40, comment="#", header=false,ignorerepeated=true, delim=' ')

```

now I wonder only if I can already write something like:

```julia
col1, col2 = CSV.File(.....)

```

---

<div class="post-metadata">

### Author: ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)
#### Post date: [January 18, 2024, 2:39pm UTC](https://discourse.julialang.org/t/read-vector-from-data-file/108945/9 "2024-01-18T14:39:05Z")

</div>

Using Tables.jl:

```julia
c1, c2 = CSV.File(...) |> Tables.columntable

```
