Copy metadata between DataFrames

Specific Domains Machine Learning
1

Is there a method to copy all of the metadata from one DataFrame to another?

2

for anyone that comes next:

md = DataFrames.metadata(df1)
for k in keys(md)
    DataFrames.metadata!(df2, k, md[k])
end
1 Like
4

Just in case you weren’t aware, there are two “styles” of metadata in Julia.

There is the :note style and the :default style.

In the :default style, metadata is dropped after every transformations. So something like

julia> df = DataFrame(a = 1);

julia> colmetadata!(df, :a, "label", "Label for a")
1×1 DataFrame
 Row │ a     
     │ Int64 
─────┼───────
   1 │     1

julia> colmetadata(df)
Dict{Symbol, Dict{String, String}} with 1 entry:
  :a => Dict("label"=>"Label for a")

julia> df2 = transform(df, :a => ByRow(t -> t + 1) => :b);

julia> colmetadata(df2)
Dict{Any, Any}()

You might be struggling with this. The solution is to set the colmetadatastyle to :note

julia> df = DataFrame(a = 1);

julia> colmetadata!(df, :a, "label", "Label for a")
1×1 DataFrame
 Row │ a     
     │ Int64 
─────┼───────
   1 │     1

julia> setcolmetadatastyle!(df; style =:note);

julia> df2 = transform(df, :a => ByRow(t -> t + 1) => :b);

julia> colmetadata(df)
Dict{Symbol, Dict{String, String}} with 1 entry:
  :a => Dict("label"=>"Label for a")

Two solutions

  1. TableMetaDataTools.jl adds some convenience functions
  2. DataFramesMeta.jl adds even more convenience functions. All metadata created with DataFramesMeta.jl is with the :note style. So they are forwarded.