Unable to save Unitful unit into a JLD file

General Usage
1

I am writing an application where the user may choose the length unit from among those provided by the Unitful package. I create a struct where the chosen length unit is stored in one of the fields. I found that when I attempted to save the struct to a .jld file using the JLD package, an error occurs. Here is a simplified MWE:

julia> using JLD, Unitful

julia> u = unit(1u"cm")
cm

julia> save("temp.jld", "u", u)
ERROR: UndefVarError: cm not defined
Stacktrace:
 [1] top-level scope at REPL[3]:1
 [2] eval at .\boot.jl:331 [inlined]
 [3] eval at C:\Users\peter\.julia\packages\JLD\jeGJb\src\JLD.jl:1 [inlined]
 [4] full_typename(::Base.GenericIOBuffer{Array{UInt8,1}}, ::JLD.JldFile, ::Tuple{Unitful.Unit{:Meter,�}}) at C:\Users\peter\.julia\packages\JLD\jeGJb\src\JLD.jl:1065
 [5] full_typename(::Base.GenericIOBuffer{Array{UInt8,1}}, ::JLD.JldFile, ::DataType) at C:\Users\peter\.julia\packages\JLD\jeGJb\src\JLD.jl:1103
 [6] full_typename(::JLD.JldFile, ::Type{T} where T) at C:\Users\peter\.julia\packages\JLD\jeGJb\src\JLD.jl:1116
 [7] commit_datatype(::JLD.JldFile, ::HDF5.HDF5Datatype, ::Any) at C:\Users\peter\.julia\packages\JLD\jeGJb\src\jld_types.jl:61
 [8] h5type_default(::JLD.JldFile, ::Any, ::Bool) at C:\Users\peter\.julia\packages\JLD\jeGJb\src\jld_types.jl:349
 [9] h5type at C:\Users\peter\.julia\packages\JLD\jeGJb\src\jld_types.jl:325 [inlined]
 [10] write_compound(::JLD.JldFile, ::String, ::Unitful.FreeUnits{(cm,),�,nothing}, ::JLD.JldWriteSession; kargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at C:\Users\peter\.julia\packages\JLD\jeGJb\src\JLD.jl:706
 [11] write_compound at C:\Users\peter\.julia\packages\JLD\jeGJb\src\JLD.jl:704 [inlined]
 [12] #_write#23 at C:\Users\peter\.julia\packages\JLD\jeGJb\src\JLD.jl:700 [inlined]
 [13] _write at C:\Users\peter\.julia\packages\JLD\jeGJb\src\JLD.jl:700 [inlined]
 [14] #write#17 at C:\Users\peter\.julia\packages\JLD\jeGJb\src\JLD.jl:524 [inlined]
 [15] write at C:\Users\peter\.julia\packages\JLD\jeGJb\src\JLD.jl:524 [inlined]
 [16] (::JLD.var"#39#40"{String,Unitful.FreeUnits{(cm,),�,nothing},Tuple{}})(::JLD.JldFile) at C:\Users\peter\.julia\packages\JLD\jeGJb\src\JLD.jl:1235
 [17] jldopen(::JLD.var"#39#40"{String,Unitful.FreeUnits{(cm,),�,nothing},Tuple{}}, ::String, ::Vararg{String,N} where N; kws::Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol,Symbol},NamedTuple{(:compatible, :compress),Tuple{Bool,Bool}}}) at C:\Users\peter\.julia\packages\JLD\jeGJb\src\JLD.jl:243
 [18] save(::FileIO.File{FileIO.DataFormat{:JLD}}, ::String, ::Unitful.FreeUnits{(cm,),�,nothing}; compatible::Bool, compress::Bool) at C:\Users\peter\.julia\packages\JLD\jeGJb\src\JLD.jl:1233
 [19] save(::FileIO.File{FileIO.DataFormat{:JLD}}, ::String, ::Unitful.FreeUnits{(cm,),�,nothing}) at C:\Users\peter\.julia\packages\JLD\jeGJb\src\JLD.jl:1230
 [20] save(::String, ::String, ::Vararg{Any,N} where N; options::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at C:\Users\peter\.julia\packages\FileIO\ZknoK\src\loadsave.jl:118
 [21] save(::String, ::String, ::Unitful.FreeUnits{(cm,),�,nothing}) at C:\Users\peter\.julia\packages\FileIO\ZknoK\src\loadsave.jl:118  [22] top-level scope at REPL[3]:1
 [23] eval(::Module, ::Any) at .\boot.jl:331
 [24] eval_user_input(::Any, ::REPL.REPLBackend) at c:\users\peter\appdata\local\programs\julia\julia-1.4.2\share\julia\stdlib\v1.4\REPL\src\REPL.jl:86
 [25] run_backend(::REPL.REPLBackend) at C:\Users\peter\.julia\packages\Revise\XFtoQ\src\Revise.jl:1162
 [26] top-level scope at none:0
 [27] eval(::Module, ::Any) at .\boot.jl:331
 [28] eval_user_input(::Any, ::REPL.REPLBackend) at c:\users\peter\appdata\local\programs\julia\julia-1.4.2\share\julia\stdlib\v1.4\REPL\src\REPL.jl:86
 [29] run_backend(::REPL.REPLBackend) at C:\Users\peter\.julia\packages\Revise\XFtoQ\src\Revise.jl:1162
 [30] top-level scope at none:0

This is with Julia v1.4.2, JLD v0.10.0, and Unitful v0.18.0.

I could extract a string representation of the unit using string(u) and save only the string to the file, but after reading the file back in, I don’t know how to convert that string back into a Unitful unit.

Any help with either the file saving/reading or string conversion to unit would be appreciated!

2

Just use Serialization

using Unitful, Serialization

u = unit(1u"cm")

serialize("tmp.jls", u)

u2 = deserialize("tmp.jls")

u == u2 # true
3

Thanks for that suggestion. I originally considered Serialization but was put off by the following warning in the documentation for serialize:

In general, this process will not work if the reading and writing are done by different
versions of Julia, or an instance of Julia with a different system image.

I would prefer a less brittle format suitable for archival purposes.

4

Sure. Jld never worked for me either. So I don’t know which is more brittle. U can always read it back with same version of Julia.

5

I just ran the code and it doesn’t seem like .jls contains the version used to create the file. I feel that would be better than knowing it was a version of julia at some point in time. I am surprised there is not a reliable binary format to save julia data.

6

I am not. I have used python and their pickle :cucumber: story is worse… So yeah.

I wonder if rds is good. But r has fewer structures and also R can’t save xgboost object either and need a specialised function for that.

Curious. U have used a tool that allows saving stuff that is cross version and safe?

7

In the past when I was heavily using matlab I used their binary format which while not ideal (not very performant) it worked across versions and operating systems. For python I have used .npy and sometimes hdf.

8

Same thing with R. I think most matlab structures are matrices and the type system is not as rich as Julia’s (if there is a type system), so for those system I imagine it’d be easier to write a serializer.

9

I experimented with BSON but it also had a (different) problem. Serialization works, so I am using it for now. Thanks for your help.

10

In case you are not awre the options are

Base.Serialization, BSON.jl, JLSO.jl, JLD.jl and JLD2.jl

For dataframes-like you have Feather.jl Parquet.jl, JDF.jl, HDF5.jl

2 Likes