# Faster groupwise joins to complete implicitly missing rows

**URL:** <https://discourse.julialang.org/t/faster-groupwise-joins-to-complete-implicitly-missing-rows/59414>\
**Category:** Performance\
**Tags:** dataframes\
**Created:** [April 16, 2021, 1:17pm UTC](https://discourse.julialang.org/t/faster-groupwise-joins-to-complete-implicitly-missing-rows/59414 "2021-04-16T13:17:39Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![danielw2904](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/danielw2904/32/10890_2.png) [@danielw2904](https://discourse.julialang.org/u/danielw2904)\
**Post date:** [April 16, 2021, 1:17pm UTC](https://discourse.julialang.org/t/faster-groupwise-joins-to-complete-implicitly-missing-rows/59414/1 "2021-04-16T13:17:39Z")

</div>

I have a grouped `DataFrame` with dates and values but only observe dates in which a value was observed. I’d like to create the rows that are implicitly missing and have them as missing e.g. for interpolation later. I came up with the following code but was wondering if there is a faster way to do this?

```julia
df = DataFrame(
    g = ['a','a', 'b', 'b', 'c', 'c', 'c'], 
    date = [Date(2021,1,1), Date(2021,1,2), Date(2021,1,2), Date(2021,1,4), Date(2021,1,1),Date(2021,1,3) ,Date(2021,1,7)],
    v = rand(7)
)
alldates = DataFrame(date = minimum(df.date):Day(1):maximum(df.date))
gdf = groupby(df, :g)
combdf = DataFrame()
for g in gdf
    gout = leftjoin(alldates, g, on = :date)
    gout.g .= g.g[1]
    disallowmissing!(gout, :g)
    append!(combdf, gout, cols = :union)
end

```

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [April 16, 2021, 2:17pm UTC](https://discourse.julialang.org/t/faster-groupwise-joins-to-complete-implicitly-missing-rows/59414/2 "2021-04-16T14:17:25Z")

</div>

Not sure if faster, but I think this is clearer:

```julia
julia> combdf2 = rename!(DataFrame(Iterators.product(alldates.date, unique(df.g))), [:date, :g])
julia> combdf2 = leftjoin(combdf2, df, on = [:date, :g])

julia> isequal(combdf, combdf2)
true

```

---

<div class="post-metadata">

**Author:** ![danielw2904](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/danielw2904/32/10890_2.png) [@danielw2904](https://discourse.julialang.org/u/danielw2904)\
**Post date:** [April 16, 2021, 9:05pm UTC](https://discourse.julialang.org/t/faster-groupwise-joins-to-complete-implicitly-missing-rows/59414/3 "2021-04-16T21:05:53Z")

</div>

Thanks! It is not only clearer but also twice as fast in a quick benchmark!