# How to add metadata info to a DataFrame?

**URL:** <https://discourse.julialang.org/t/how-to-add-metadata-info-to-a-dataframe/11168>\
**Category:** Data\
**Tags:** dataframes, metadata\
**Created:** [May 26, 2018, 3:21pm UTC](https://discourse.julialang.org/t/how-to-add-metadata-info-to-a-dataframe/11168 "2018-05-26T15:21:35Z")\
**Posts on this page:** 1\
**Showing post:** 27

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [May 27, 2018, 4:12pm UTC](https://discourse.julialang.org/t/how-to-add-metadata-info-to-a-dataframe/11168/27 "2018-05-27T16:12:13Z")

</div>

I agree both solutions are reasonable: store the meta-data in the data frame, or in each column vector. Each has advantages and drawbacks:

- In the data frame:
  - Advantages: columns are just standard `Vector` objects, which is simpler for users (it feels kind of weird to avoid using the standard array type in very common use cases) and to implement
  - Drawbacks: meta-data is lost as soon as a column is used separately from its data frame

- In the column vector:
  - Advantages: meta-data is kept with the data it describes and some functions could make use of that even without supporting data frames
  - Drawbacks: requires using a custom array type; can become quite involved if you need a `MetaDataVector{CategoricalVector{...}}` (which will be quite common). For example, we’ll need custom `recode` methods which call the efficient `CategoricalArray` method instead of the fallback `AbstractArray` one. Any custom array type may need similar tricks, which (currently) requires one package to depend on the other.

It seems that dplyr has chosen the second approach ([see](https://github.com/tidyverse/tibble/issues/66) [these](https://github.com/tidyverse/tibble/issues/155) [issues](https://github.com/tidyverse/tibble/issues/156)). In particular, [`haven::read_dta`](https://www.rdocumentation.org/packages/haven/versions/1.1.1/topics/read_dta) stores meta-data from Stata as attributes of the column vectors. The [memisc](https://cran.r-project.org/web/packages/memisc/vignettes/items.html), [Hmisc](https://www.rdocumentation.org/packages/Hmisc/versions/4.1-1/topics/label) and [labelled](https://cran.r-project.org/web/packages/labelled/vignettes/intro_labelled.html) do the same. But of course R is much more limited in terms of dispatch, so the situation is quite different.

Something we should also think about is how meta-data could be preserved across streaming operations with Query and DataStreams. `CategoricalArray` handles this via the special `CategoricalValue` element type, but a `MetaDataArray` just contains normal entries, so you can’t retrieve the meta-data from individual entries. I’m not sure which of the two ways of storing meta-data would be easier to handle with Query and DataStreams. This matters in particular when combining Query with plots à la [StatsPlots](https://github.com/JuliaPlots/StatPlots.jl). We should probably resolve this issue before choosing what’s the most appropriate approach. Maybe an extension of schemas to include meta-data in addition to column names and types would be useful; in that case either approach would be OK. Cc: @davidanthoff @mkborregaard @quinnj .

---

_[View the full topic](https://discourse.julialang.org/t/how-to-add-metadata-info-to-a-dataframe/11168)._