# DataFrame in Nested Loop

**URL:** <https://discourse.julialang.org/t/dataframe-in-nested-loop/51046>\
**Category:** New to Julia\
**Tags:** dataframes\
**Created:** [December 1, 2020, 1:36pm UTC](https://discourse.julialang.org/t/dataframe-in-nested-loop/51046 "2020-12-01T13:36:43Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rojesh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rojesh/32/15390_2.png) [@Rojesh](https://discourse.julialang.org/u/Rojesh)\
**Post date:** [December 1, 2020, 1:36pm UTC](https://discourse.julialang.org/t/dataframe-in-nested-loop/51046/1 "2020-12-01T13:36:43Z")

</div>

I’m trying to append data to a dataframe through nested loop something similar to the following example, but the resulting dataframe is empty. How can I fix this ?  
I’ll be calling another function instead of DataFrame that has argument year and month, which doesn’t have the issue.

```julia
df = DataFrame()
for year in 1993:2020, month in 1:12
    append!(df,DataFrame(A=year,B=month))
end
df

```

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [December 1, 2020, 1:59pm UTC](https://discourse.julialang.org/t/dataframe-in-nested-loop/51046/2 "2020-12-01T13:59:21Z")

</div>

It seems to work here:

```julia
julia> using DataFrames

julia> df = DataFrame()
0×0 DataFrame

julia> for year in 1993:2020, month in 1:12
           append!(df,DataFrame(A=year,B=month))
       end

julia> df
336×2 DataFrame
│ Row │ A │ B │
│ │ Int64 │ Int64 │
├─────┼───────┼───────┤
│ 1 │ 1993 │ 1 │
│ 2 │ 1993 │ 2 │
│ 3 │ 1993 │ 3 │
│ 4 │ 1993 │ 4 │
│ 5 │ 1993 │ 5 │

```

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [December 1, 2020, 2:39pm UTC](https://discourse.julialang.org/t/dataframe-in-nested-loop/51046/3 "2020-12-01T14:39:41Z")

</div>

Side note, you probably want `push!`

```julia

julia> df = DataFrame()
0×0 DataFrame

julia> for year in 1993:2020, month in 1:12
           push!(df,(A=year,B=month))
       end

```

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [December 1, 2020, 2:45pm UTC](https://discourse.julialang.org/t/dataframe-in-nested-loop/51046/4 "2020-12-01T14:45:59Z")

</div>

To make it more precise 😄 : both `append!` and `push!` are OK, but `push!` will be faster. Here is a comparison:

```julia
julia> function f1()
       df = DataFrame()
       for year in 1993:2020, month in 1:12
           append!(df, DataFrame(A=year,B=month))
       end
       return df
       end
f1 (generic function with 1 method)

julia> function f2()
       df = DataFrame()
       for year in 1993:2020, month in 1:12
           push!(df, (A=year,B=month))
       end
       return df
       end
f2 (generic function with 1 method)

julia> function f3()
       df = DataFrame(A=Int[], B=Int[])
       for year in 1993:2020, month in 1:12
           push!(df, (year,month))
       end
       return df
       end
f3 (generic function with 1 method)

julia> @time f1();
  0.001682 seconds (13.13 k allocations: 1020.641 KiB)

julia> @time f2();
  0.000212 seconds (1.71 k allocations: 81.266 KiB)

julia> @time f3();
  0.000123 seconds (378 allocations: 23.922 KiB)

```

(timings are after compilation)

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [December 1, 2020, 2:49pm UTC](https://discourse.julialang.org/t/dataframe-in-nested-loop/51046/5 "2020-12-01T14:49:52Z")

</div>

And finally, just consider using the excellent `Dates` standard library for this:

```julia
julia> using Dates, DataFrames

julia> df = DataFrame(date = Date(1993,1):Month(1):Date(2020, 12));

julia> df.year = year.(df.date); df.month = month.(df.date);

julia> first(df, 5)
5×3 DataFrame
 Row │ date year month 
     │ Date Int64 Int64 
─────┼──────────────────────────
   1 │ 1993-01-01 1993 1
   2 │ 1993-02-01 1993 2
   3 │ 1993-03-01 1993 3
   4 │ 1993-04-01 1993 4
   5 │ 1993-05-01 1993 5

```

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [December 1, 2020, 2:53pm UTC](https://discourse.julialang.org/t/dataframe-in-nested-loop/51046/6 "2020-12-01T14:53:17Z")

</div>

As a side note - thanks to @Ronis_BR it is extremely easy visually to check which version of DataFrames.jl someone is working on (as this matters in some cases - not this time fortunately). 😃

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [December 1, 2020, 3:10pm UTC](https://discourse.julialang.org/t/dataframe-in-nested-loop/51046/7 "2020-12-01T15:10:53Z")

</div>

> [@pdeffebach](#):
>
> Side note, you probably want `push!`

This is probably not related to DataFrames, but why `push` and `append` are any different in this case? If one of those are faster, shouldn’t the methods called at the end be the same?

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [December 1, 2020, 3:14pm UTC](https://discourse.julialang.org/t/dataframe-in-nested-loop/51046/8 "2020-12-01T15:14:20Z")

</div>

`push!` is for single elements, in this case pushing a “row” (`NamedTuple`) to a collection of rows (`DataFrame`). `append!` is for concatenating collections (`DataFrame`s), so you can `append!` two data frames.

The issue is not the speed of the `append!` it’s the cost of constructing a DataFrame each time in the loop, which you have to do in order to `append!`.

---

<div class="post-metadata">

**Author:** ![Rojesh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rojesh/32/15390_2.png) [@Rojesh](https://discourse.julialang.org/u/Rojesh)\
**Post date:** [December 3, 2020, 2:53am UTC](https://discourse.julialang.org/t/dataframe-in-nested-loop/51046/9 "2020-12-03T02:53:23Z")

</div>

thanks, I didn’t know dates could be used in this way. Also it would be faster compared to append! or push!.