# Looking for a clean way to add a row in a DataFrame

**URL:** <https://discourse.julialang.org/t/looking-for-a-clean-way-to-add-a-row-in-a-dataframe/123891>\
**Category:** General Usage\
**Tags:** question, dataframes, csv\
**Created:** [December 16, 2024, 3:20pm UTC](https://discourse.julialang.org/t/looking-for-a-clean-way-to-add-a-row-in-a-dataframe/123891 "2024-12-16T15:20:14Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![dobby](https://avatars.discourse-cdn.com/v4/letter/d/7993a0/32.png) [@dobby](https://discourse.julialang.org/u/dobby)\
**Post date:** [December 16, 2024, 3:20pm UTC](https://discourse.julialang.org/t/looking-for-a-clean-way-to-add-a-row-in-a-dataframe/123891/1 "2024-12-16T15:20:14Z")

</div>

Hello,

I am new to Julia, so please have mercy.  
I am reading multiple CSV files, where the rows are the same. But in order to distinguish the different files I’m reading, I have to add a row to indicate that.  
Here is a minimalistic example:

```julia
header = ["Col1", "Col2", "Set_number"]
observation1 = [1, 2]
observation2 = [3, 4]
# ...
observationX = [7, 8]
relevant_data = DataFrame([[], [], []], header)

# sets is an array with filepaths
for set in sets
        filtered_data = CSV.read(set, header)
        filtered_data.Set .= set_number
        append!(relevant_data, filtered_data)
    end
end

```

So in order to add the missing row, I am reading the files intentionally with a given header, that is longer than the data set. The missing data is filled with “missing” and I replace it later with the intentional value, depending on the current set in the loop.

This leads to the problem, that the terminal is spammed with the warning that missing data is filled in.

Is there a neater way to do what I want to do?  
It feels like I am not doing it quite right, although it works for me just fine.

---

<div class="post-metadata">

**Author:** ![g-gundam](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/g-gundam/32/47593_2.png) [@g-gundam](https://discourse.julialang.org/u/g-gundam)\
**Post date:** [December 16, 2024, 3:39pm UTC](https://discourse.julialang.org/t/looking-for-a-clean-way-to-add-a-row-in-a-dataframe/123891/2 "2024-12-16T15:39:34Z")

</div>

What happens to those `observationN` variables?

---

<div class="post-metadata">

**Author:** ![algunion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/algunion/32/51630_2.png) [@algunion](https://discourse.julialang.org/u/algunion)\
**Post date:** [December 16, 2024, 3:44pm UTC](https://discourse.julialang.org/t/looking-for-a-clean-way-to-add-a-row-in-a-dataframe/123891/3 "2024-12-16T15:44:52Z")

</div>

The clean version would be to add the column `Set_number` to your `filtered_data` data frame and append the `filtered_data` to `relevant_data` after this step.

This will solve the warning issue (and it seems compatible with the logic in your `for` loop).

P. S. You seem to be already on the right path, but you assign the value to the `Set` instead of `Set_number` (see the @pdeffebach comment below).

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [December 16, 2024, 3:44pm UTC](https://discourse.julialang.org/t/looking-for-a-clean-way-to-add-a-row-in-a-dataframe/123891/4 "2024-12-16T15:44:59Z")

</div>

I’m a little confused, but

```julia
        filtered_data.Set .= set_number

```

contradicts that you named the column `Set_number` in the beginning.

Maybe that’s your issue?

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [December 16, 2024, 3:47pm UTC](https://discourse.julialang.org/t/looking-for-a-clean-way-to-add-a-row-in-a-dataframe/123891/5 "2024-12-16T15:47:47Z")

</div>

Given `data.csv`:

```julia
a,b,c
1,2,3
4,5,6

```

You can do this:

```julia
CSV.read(
    ["data.csv", "data.csv"],
    DataFrame;
    source = :Set => [1, 2]
)

```

to get

```julia
4×4 DataFrame
 Row │ a b c Set   
     │ Int64 Int64 Int64 Int64 
─────┼────────────────────────────
   1 │ 1 2 3 1
   2 │ 4 5 6 1
   3 │ 1 2 3 2
   4 │ 4 5 6 2

```

---

<div class="post-metadata">

**Author:** ![dobby](https://avatars.discourse-cdn.com/v4/letter/d/7993a0/32.png) [@dobby](https://discourse.julialang.org/u/dobby)\
**Post date:** [December 17, 2024, 8:10am UTC](https://discourse.julialang.org/t/looking-for-a-clean-way-to-add-a-row-in-a-dataframe/123891/6 "2024-12-17T08:10:56Z")

</div>

Thank you everyone!  
Sorry that my example was a bit confusing. I didn’t find how to edit it.  
Jules gave a beautiful solution 🙂

---

<div class="post-metadata">

**Author:** ![g-gundam](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/g-gundam/32/47593_2.png) [@g-gundam](https://discourse.julialang.org/u/g-gundam)\
**Post date:** [December 17, 2024, 2:47pm UTC](https://discourse.julialang.org/t/looking-for-a-clean-way-to-add-a-row-in-a-dataframe/123891/7 "2024-12-17T14:47:32Z")

</div>

> [@dobby](#):
>
> I didn’t find how to edit it.

![image](https://global.discourse-cdn.com/julialang/original/3X/f/c/fc7c3c99a6073140beaee5b83b3f33d0c554add1.png)

Clicking on that little pencil icon at the bottom of your post will allow you to edit it.

---

<div class="post-metadata">

**Author:** ![dobby](https://avatars.discourse-cdn.com/v4/letter/d/7993a0/32.png) [@dobby](https://discourse.julialang.org/u/dobby)\
**Post date:** [December 17, 2024, 3:29pm UTC](https://discourse.julialang.org/t/looking-for-a-clean-way-to-add-a-row-in-a-dataframe/123891/8 "2024-12-17T15:29:26Z")

</div>

That’s the weird thing, I don’t have it ☹

 ![image](https://global.discourse-cdn.com/julialang/original/3X/9/d/9d236c0e3f3de49afe525c97effb3f6e6eb81589.png)

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [December 17, 2024, 4:15pm UTC](https://discourse.julialang.org/t/looking-for-a-clean-way-to-add-a-row-in-a-dataframe/123891/9 "2024-12-17T16:15:56Z")

</div>

```julia
julia> vcat((CSV.read.(["f1.csv", "f2.csv"],DataFrame))..., source=:sour=>[:f1,:f2])
4×4 DataFrame
 Row │ a b c sour   
     │ Int64 Int64 Int64 Symbol
─────┼─────────────────────────────
   1 │ 1 2 3 f1
   2 │ 4 5 6 f1
   3 │ 11 21 31 f2
   4 │ 41 51 61 f2

```

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [December 18, 2024, 1:52pm UTC](https://discourse.julialang.org/t/looking-for-a-clean-way-to-add-a-row-in-a-dataframe/123891/10 "2024-12-18T13:52:18Z")

</div>

```julia
julia> f1="""a,b,c
       1,2,3
       4,5,6"""
"a,b,c\n1,2,3\n4,5,6"

julia> f2="""a,c,d
       11,21,31
       41,51,61"""
"a,c,d\n11,21,31\n41,51,61"

julia> f3="""a,b,d
       11,21,31
       41,51,61"""
"a,b,d\n11,21,31\n41,51,61"

julia> write("f1.csv", f1)
17

julia> write("f2.csv", f2)
23

julia> write("f3.csv", f3)
23

julia> using DataFrames, CSV

julia> vcat((CSV.read.(["f1.csv", "f2.csv", "f3.csv"],DataFrame))...,cols=:union, source=:sour=>[:f1,:f2,:f3]) 
6×5 DataFrame
 Row │ a b c d sour   
     │ Int64 Int64? Int64? Int64? Symbol 
─────┼──────────────────────────────────────────
   1 │ 1 2 3 missing f1
   2 │ 4 5 6 missing f1
   3 │ 11 missing 21 31 f2
   4 │ 41 missing 51 61 f2
   5 │ 11 21 missing 31 f3
   6 │ 41 51 missing 61 f3

julia> CSV.read(
           ["f1.csv", "f2.csv", "f3.csv"],
           DataFrame;
           source = :Set => Symbol.("f".*string.(1:3)))
6×4 DataFrame
 Row │ a b c Set    
     │ Int64 Int64? Int64? Symbol
─────┼─────────────────────────────────
   1 │ 1 2 3 f1
   2 │ 4 5 6 f1
   3 │ 11 missing 21 f2
   4 │ 41 missing 51 f2
   5 │ 11 21 missing f3
   6 │ 41 51 missing f3

```

Just to point out that in a case like this using CSV alone does not provide the expected result.  
I don’t know if there is a different form of CSV.read() that can handle this situation.

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [December 18, 2024, 2:44pm UTC](https://discourse.julialang.org/t/looking-for-a-clean-way-to-add-a-row-in-a-dataframe/123891/11 "2024-12-18T14:44:24Z")

</div>

> [@dobby](#):
>
> That’s the weird thing, I don’t have it ☹

I think you can’t edit when you’re this new to the community (welcome by the way!). It’s an anti-spam thing. Don’t sweat it! It was clear enough that you got to an answer 🙂

> [@dobby](#):
>
> I am new to Julia, so please have mercy.

Don’t worry, people here tend to be quite kind, as long as you don’t toss bombs and are making an effort 😉