How to easily save Julia data?

General Usage
1

Hi guys,
I am wondering is there a way like pickle in python to save the output data from Julia calculations? maybe it can support arrays, dict, tuples? for example, if I have a dict of which each key stores a different dataframe (columns rows are different), then how can I save it? and then how to load it? (if the method is compatible with python, then it would be much more better!)

Here is my example

julia> sensitivity_results
Dict{Any, Any} with 230 entries:
  "pWaterRootHalfLoss_transpiration"         => 1Γ—4 DataFrame…
  "ΞΊ_total_plant_water_content"              => 1Γ—4 DataFrame…
  ....
julia> sensitivity_results["pWaterRootHalfLoss_transpiration"]
1Γ—4 DataFrame
 Row β”‚ sensitivity_mean  sensitivity_std  sensitivity_daily  sensitivity_seasonal 
     β”‚ Float64           Float64          Float64            Float64              
─────┼────────────────────────────────────────────────────────────────────────────
   1 β”‚      0.000450049      0.000635199        0.000538933           0.000614739
julia> sensitivity_results["ΞΊ_total_plant_water_content"]
1Γ—4 DataFrame
 Row β”‚ sensitivity_mean  sensitivity_std  sensitivity_daily  sensitivity_seasonal 
     β”‚ Float64           Float64          Float64            Float64              
─────┼────────────────────────────────────────────────────────────────────────────
   1 β”‚        -0.131991        -0.148735          -0.146674             -0.148216

Thanks!

1 Like
2

In my experience JLD2.jl is equivalent to the pickle module in python. To get python compatible objects you can save tabular data with Arrow.jl or simply write a CSV.

6 Likes
3

While I would also suggest JLD2.jl to save Julia data, the equivalent to the pickle module of Python is Serialization Β· The Julia Language , which is part of Julia and does not require an extra package. You still need to write using Serialization at the top of your file, though.

3 Likes
4

BSON.jl and JLSO.jl also work quite well. You can save complete structs and just need to import the corresponding module when you want to deserialize it.

2 Likes
5

Serialization is the most convenient one.

6

Well, all solutions have advantages and disadvantages. Look at:

  • backwards compatibility (can a new version read data written by an old version)
  • do you just want to save just numbers and strings, or more complex - perhaps even Julia specific - data structures
  • performance (read, write and package load time)
  • file size
  • shall the data format be standardized, shall it be possible to read and write it in other programming languages or on computers with a different CPU architecture

If you know all of that, then you can choose the best solution for your use case.

5 Likes