# Best way to import a CSV with a repeated structure

**URL:** <https://discourse.julialang.org/t/best-way-to-import-a-csv-with-a-repeated-structure/119377>\
**Category:** Data\
**Tags:** io\
**Created:** [September 13, 2024, 3:44pm UTC](https://discourse.julialang.org/t/best-way-to-import-a-csv-with-a-repeated-structure/119377 "2024-09-13T15:44:21Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![Dan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dan/32/42581_2.png) [@Dan](https://discourse.julialang.org/u/Dan)\
**Post date:** [September 14, 2024, 8:20pm UTC](https://discourse.julialang.org/t/best-way-to-import-a-csv-with-a-repeated-structure/119377/4 "2024-09-14T20:20:44Z")

</div>

Suppose your file looks like:

```julia
$ cat in.txt 
Day 1
1	10
2	20
3	15
4	20

Day 2
1	25
2	30
3	40
4	35

Day 3
1	40
2	45
3	30
4	15

```

A little Unix shell magic can be used to help Julia parse this as follows:

```julia
julia> using CSV, DataFrames

julia> CSV.read(pipeline(`awk 'BEGIN { day = "0"; print "day\ttime\tvalue" } /^Day/ { day = $2} /^[^D]/ {print day,"\t" $0}' in.txt`), DataFrame; delim='\t')
12×3 DataFrame
 Row │ day time value 
     │ Int64 Int64 Int64 
─────┼─────────────────────
   1 │ 1 1 10
   2 │ 1 2 20
   3 │ 1 3 15
   4 │ 1 4 20
   5 │ 2 1 25
   6 │ 2 2 30
   7 │ 2 3 40
   8 │ 2 4 35
   9 │ 3 1 40
  10 │ 3 2 45
  11 │ 3 3 30
  12 │ 3 4 15

```

The key useful info is the way `awk` tool is used to separate the important data lines from the intra-file headers, and to process the data in the header to enable adding it later to each line separately. This is the `awk` command on its own:

```julia
awk 'BEGIN { day = "0"; print "day\ttime\tvalue" } /^Day/ { day = $2} /^[^D]/ {print day,"\t" $0}' in.txt

```

Hopefully the awk logic is clear enough.

P.S. screenshots are annoying in a post. Try to insert copy-pasteable stuff

---

_[View the full topic](https://discourse.julialang.org/t/best-way-to-import-a-csv-with-a-repeated-structure/119377)._