# Fill missing kwarg in vcat function when cols = :union

**URL:** <https://discourse.julialang.org/t/fill-missing-kwarg-in-vcat-function-when-cols-union/78214>\
**Category:** Data\
**Tags:** missing-values\
**Created:** [March 21, 2022, 2:26pm UTC](https://discourse.julialang.org/t/fill-missing-kwarg-in-vcat-function-when-cols-union/78214 "2022-03-21T14:26:06Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [March 21, 2022, 2:26pm UTC](https://discourse.julialang.org/t/fill-missing-kwarg-in-vcat-function-when-cols-union/78214/1 "2022-03-21T14:26:06Z")

</div>

I am submitting to you a question which I believe to be of a general nature (along with other issues arising from the case), derived from a particular case on which I was working.  
I find myself in the need to make transformations (via `reshapein (...)`) of the various subdataframes obtained from `groupby (df, ...)` and then combine them together with the use of vcat.  
Since applying the `reshapein (...)` function to the various subgroups generates dataframes with a different set of columns, I cannot use `combine (groupby (...), ...),` although an option would be useful in these cases like that of `vcat cols =: union.`  
Therefore I regret merging with `vcat (..., cols =: union)`. I wonder if it wasn’t useful in cases like these where I know well the origin of the various missing items, if it wasn’t useful to have a `fillmissing = something` argument.  
In the absence of this I have defined and used the two functions `filldown ()` and `fillup ()` which replace the missing value with the last useful value available forward or backward.

Also with regard to functions of this type I believe that they are quite of general necessity and it would not be bad if they were available (not in naive form, like the one made by me) in the dataframes.jl package

```julia
filldown(v)=accumulate((x,y)->coalesce(y,x), v,init=v[1])

fillup(v)=reverse(filldown(reverse(v)))

function reshape(grp)
    sgrps=groupby(df[grp,:],:Page)
    rr=mapreduce(reshapein,(x,y)-> vcat(x,y, cols = :union), sgrps)
    transform!(rr,Cols(:).=> filldown∘fillup,renamecols=false)
end

```

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [March 21, 2022, 2:35pm UTC](https://discourse.julialang.org/t/fill-missing-kwarg-in-vcat-function-when-cols-union/78214/2 "2022-03-21T14:35:11Z")

</div>

If I understand you correctly, `filldown` and `fillup` are “last observation carried forward” and “next observation carried backward” in Impute.jl

[https://invenia.github.io/Impute.jl/stable/api/imputation/#Last-Observation-Carried-Forward-(LOCF)](https://invenia.github.io/Impute.jl/stable/api/imputation/#Last-Observation-Carried-Forward-(LOCF))

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [March 22, 2022, 8:04am UTC](https://discourse.julialang.org/t/fill-missing-kwarg-in-vcat-function-when-cols-union/78214/3 "2022-03-22T08:04:14Z")

</div>

What do you think about the idea of having a kwarg in cases like this in `vcat(....; cols= :union, fill = ...)` ?

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [March 22, 2022, 8:39am UTC](https://discourse.julialang.org/t/fill-missing-kwarg-in-vcat-function-when-cols-union/78214/4 "2022-03-22T08:39:45Z")

</div>

I would personally be opposed to it, as it would complicate the DataFrames API (which is already quite large) and reimplement functionality that is better placed in a dedicated package (`Impute.jl` offers loads more than just those two options), leveraging Julia’s modularity.

That said I’m not a maintainer, so feel free to open a PR with a feature request.