# Saving Unitful DataFrame to file

**URL:** https://discourse.julialang.org/t/saving-unitful-dataframe-to-file/69313
**Category:** Data
**Tags:** dataframes, unitful
**Created:** [October 6, 2021, 3:58pm UTC](https://discourse.julialang.org/t/saving-unitful-dataframe-to-file/69313 "2021-10-06T15:58:09Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![Gregstrq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gregstrq/32/20620_2.png) [@Gregstrq](https://discourse.julialang.org/u/Gregstrq)
#### Post date: [October 6, 2021, 3:58pm UTC](https://discourse.julialang.org/t/saving-unitful-dataframe-to-file/69313/1 "2021-10-06T15:58:09Z")

</div>

Imagine I have a DataFrame with some of the columns being Unitful. Is there a way to save such a DataFrame into a file while preserving the information about the units?

The only solution I can think about is to strip it off units and save it to a .csv file. Then, when I load the file, I will have to multiply the columns by units.

But may be there is a good established approach to this?

---

<div class="post-metadata">

### Author: ![DrPapa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/drpapa/32/6835_2.png) [@DrPapa](https://discourse.julialang.org/u/DrPapa)
#### Post date: [October 6, 2021, 4:05pm UTC](https://discourse.julialang.org/t/saving-unitful-dataframe-to-file/69313/2 "2021-10-06T16:05:44Z")

</div>

Does it need to be human readable? If not , `Serialization.serialize` works out of the box for DataFrames + Unitful. [Serialization · The Julia Language](https://docs.julialang.org/en/v1/stdlib/Serialization/#Serialization.serialize)

---

<div class="post-metadata">

### Author: ![Gregstrq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gregstrq/32/20620_2.png) [@Gregstrq](https://discourse.julialang.org/u/Gregstrq)
#### Post date: [October 6, 2021, 4:15pm UTC](https://discourse.julialang.org/t/saving-unitful-dataframe-to-file/69313/3 "2021-10-06T16:15:23Z")

</div>

No, it does not have to be human readable, but I would like loading the file to be fast.

---

<div class="post-metadata">

### Author: ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)
#### Post date: [October 6, 2021, 4:20pm UTC](https://discourse.julialang.org/t/saving-unitful-dataframe-to-file/69313/4 "2021-10-06T16:20:30Z")

</div>

serialization is pretty good then

---

<div class="post-metadata">

### Author: ![Gregstrq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gregstrq/32/20620_2.png) [@Gregstrq](https://discourse.julialang.org/u/Gregstrq)
#### Post date: [October 6, 2021, 4:23pm UTC](https://discourse.julialang.org/t/saving-unitful-dataframe-to-file/69313/5 "2021-10-06T16:23:06Z")

</div>

By the way, does saving to .jld2 works as well?

---

<div class="post-metadata">

### Author: ![DrPapa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/drpapa/32/6835_2.png) [@DrPapa](https://discourse.julialang.org/u/DrPapa)
#### Post date: [October 6, 2021, 6:15pm UTC](https://discourse.julialang.org/t/saving-unitful-dataframe-to-file/69313/6 "2021-10-06T18:15:15Z")

</div>

I remember trying various serialization solutions and found Serialization to work best in terms of speed and serializing custom structs. Plus its a standard lib, which is a plus

---

<div class="post-metadata">

### Author: ![DrPapa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/drpapa/32/6835_2.png) [@DrPapa](https://discourse.julialang.org/u/DrPapa)
#### Post date: [October 6, 2021, 6:22pm UTC](https://discourse.julialang.org/t/saving-unitful-dataframe-to-file/69313/7 "2021-10-06T18:22:13Z")

</div>

For reference, a 450MB serialized dataframe file takes 109 ms to deserialize using Serialization.

---

<div class="post-metadata">

### Author: ![Raf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raf/32/3383_2.png) [@Raf](https://discourse.julialang.org/u/Raf)
#### Post date: [October 6, 2021, 6:45pm UTC](https://discourse.julialang.org/t/saving-unitful-dataframe-to-file/69313/8 "2021-10-06T18:45:51Z")

</div>

Everyone wants this (for the repl and e.g. writing to csv):  
[https://github.com/PainterQubits/Unitful.jl/issues/412](https://github.com/PainterQubits/Unitful.jl/issues/412)  
[https://github.com/PainterQubits/Unitful.jl/issues/388](https://github.com/PainterQubits/Unitful.jl/issues/388)  
[https://github.com/PainterQubits/Unitful.jl/pull/298](https://github.com/PainterQubits/Unitful.jl/pull/298)  
[https://github.com/PainterQubits/Unitful.jl/issues/391](https://github.com/PainterQubits/Unitful.jl/issues/391)  
[https://github.com/PainterQubits/Unitful.jl/issues/466](https://github.com/PainterQubits/Unitful.jl/issues/466)

There is a PR too:  
[https://github.com/PainterQubits/Unitful.jl/pull/470](https://github.com/PainterQubits/Unitful.jl/pull/470)

---

<div class="post-metadata">

### Author: ![phlavenk](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/phlavenk/32/1873_2.png) [@phlavenk](https://discourse.julialang.org/u/phlavenk)
#### Post date: [February 10, 2023, 7:08am UTC](https://discourse.julialang.org/t/saving-unitful-dataframe-to-file/69313/9 "2023-02-10T07:08:03Z")

</div>

There is a package for **reversible** converting :  
[UnitfulParsableString.jl](https://github.com/michikawa07/UnitfulParsableString.jl)  
I’m using these functions for Arrows saving using that package:

```julia
function deunitful(df::AbstractDataFrame)
    for colname in names(df)        
        col = df[!,colname]
        firstval = skipmissing(col)[1]
        if firstval isa Number
            ustr = string(Unitful.unit(firstval))
            df[!,colname] .= ustrip.(df[!,colname])
            colmetadata!(df, colname, "units", ustr, style=:note);
            colmetadata(df, colname, "units")
        end
    end
    df
end

function reunitful(df::AbstractDataFrame)
    for colname in names(df)        
        if "units" in colmetadatakeys(df, colname)        
            ustr = colmetadata(df, colname, "units")
            un = Unitful.uparse(ustr)
            df[!,colname] .= df[!,colname] .* un
            deletecolmetadata!(df, colname, "units")
        end
    end
    df
end

```
