# Recommended Saves and Loads of DataFrame : JLD, CSV, etc

**URL:** <https://discourse.julialang.org/t/recommended-saves-and-loads-of-dataframe-jld-csv-etc/9074>\
**Category:** Data\
**Created:** [February 15, 2018, 3:03am UTC](https://discourse.julialang.org/t/recommended-saves-and-loads-of-dataframe-jld-csv-etc/9074 "2018-02-15T03:03:06Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![ivo\_welch](https://avatars.discourse-cdn.com/v4/letter/i/d2c977/32.png) [@ivo\_welch](https://discourse.julialang.org/u/ivo_welch)\
**Post date:** [February 15, 2018, 3:03am UTC](https://discourse.julialang.org/t/recommended-saves-and-loads-of-dataframe-jld-csv-etc/9074/1 "2018-02-15T03:03:07Z")

</div>

I am using 0.6.2, but am planning to move to 1.0 when available.

I read in an earlier discussion (/suggested-formats-for-saving-and-serialization/3230) that JLD has not been working. I have tried to `save()` some dataframes to a jld file and then `load()` them, but this gave back junk.

I was also not clear about how to read and write dataframes in csv format. readtable and writetable seem to be working, but claim to be deprecated, and CSV was not working yet.

are these cases [a] where it is my problem, because I fail to understand how this should be done; or [b] is it better to wait a little bit for 1.0 before undertaking more experimentation.

advice appreciated.

/iaw

---

<div class="post-metadata">

**Author:** ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Post date:** [February 15, 2018, 3:45am UTC](https://discourse.julialang.org/t/recommended-saves-and-loads-of-dataframe-jld-csv-etc/9074/2 "2018-02-15T03:45:01Z")

</div>

There are many options 🙂 One family of packages you can try are the ones I created: [https://github.com/davidanthoff/CSVFiles.jl](https://github.com/davidanthoff/CSVFiles.jl), [https://github.com/davidanthoff/FeatherFiles.jl](https://github.com/davidanthoff/FeatherFiles.jl), [https://github.com/davidanthoff/StatFiles.jl](https://github.com/davidanthoff/StatFiles.jl), [https://github.com/davidanthoff/ExcelFiles.jl](https://github.com/davidanthoff/ExcelFiles.jl) and [https://github.com/davidanthoff/ParquetFiles.jl](https://github.com/davidanthoff/ParquetFiles.jl). All of them should work (and are tested nightly by a build bot), please do let me know if you run into any issues.

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [February 15, 2018, 6:30am UTC](https://discourse.julialang.org/t/recommended-saves-and-loads-of-dataframe-jld-csv-etc/9074/3 "2018-02-15T06:30:17Z")

</div>

Didn’t know about ParquetFiles.jl! Thought that I can’t save ParquetFile.jl

---

<div class="post-metadata">

**Author:** ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Post date:** [February 15, 2018, 6:58am UTC](https://discourse.julialang.org/t/recommended-saves-and-loads-of-dataframe-jld-csv-etc/9074/4 "2018-02-15T06:58:03Z")

</div>

You still can’t, ParquetFiles is just the FileIO and iterable tables integration, under the hood it uses Parquet.jl, so no saving at this point.

---

<div class="post-metadata">

**Author:** ![pasha](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pasha/32/3319_2.png) [@pasha](https://discourse.julialang.org/u/pasha)\
**Post date:** [February 15, 2018, 3:45pm UTC](https://discourse.julialang.org/t/recommended-saves-and-loads-of-dataframe-jld-csv-etc/9074/5 "2018-02-15T15:45:27Z")

</div>

Feather is nice, fast, compact, portable, and type-safe for reading and writing.

I found that pre-DataFrames 0.11, CSV is in a bit of a Bardo land. At first I was hoping to use CSV.jl, but it couldn’t remove padding, so I went back to DataFrames.readtable(). That function, readtable(), is soon to be deprecated with 0.11, so everything will change in a few months, then become amazingly useful soon after.

Your strategy may be similar - do something now that works, until DataFrames 0.11 settles down and then “finalize” your method.

---

<div class="post-metadata">

**Author:** ![ivo\_welch](https://avatars.discourse-cdn.com/v4/letter/i/d2c977/32.png) [@ivo\_welch](https://discourse.julialang.org/u/ivo_welch)\
**Post date:** [February 15, 2018, 4:27pm UTC](https://discourse.julialang.org/t/recommended-saves-and-loads-of-dataframe-jld-csv-etc/9074/6 "2018-02-15T16:27:53Z")

</div>

is there a clear path? pasha, I think you are telling me “not yet, hold on just a little longer.”

david, thanks for the links. I can probably use them for now. I would love to go official asap, though.

rant: I know it is early, but I am a bit annoyed that DataFrames, or something like it, is not a first-tier package that comes with and is officially supported by julia as _the_ standard for handling, reading, and writing data sets. If it was just me, this would not be an issue. but I need to recommend and explain a software package to our students. in a sense, data sets are more important to my students than enumerators and a lot of other first-tier aspects of julia.

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [February 15, 2018, 7:40pm UTC](https://discourse.julialang.org/t/recommended-saves-and-loads-of-dataframe-jld-csv-etc/9074/7 "2018-02-15T19:40:07Z")

</div>

If you are only talking about DataFrames then I think writing them to CSV works quite well. I’ve rarely had problems there. I use `CSV.write("filename.csv",df)` frequently from CSV.jl on Julia v0.6.2. The text parsers do a pretty good job of correctly typing the columns.

If you are talking about custom types, then it is a bit more complicated. Feather and JLD2 should be working though.

---

<div class="post-metadata">

**Author:** ![hdavid16](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hdavid16/32/11531_2.png) [@hdavid16](https://discourse.julialang.org/u/hdavid16)\
**Post date:** [August 29, 2020, 4:28pm UTC](https://discourse.julialang.org/t/recommended-saves-and-loads-of-dataframe-jld-csv-etc/9074/8 "2020-08-29T16:28:30Z")

</div>

CSV works great, but it has issues with large dataframes (see [https://github.com/JuliaData/CSV.jl/issues/735#issue-688559890](https://github.com/JuliaData/CSV.jl/issues/735#issue-688559890)). JLD2 on the otherhand works great for me with large dataframes.

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [August 30, 2020, 3:36am UTC](https://discourse.julialang.org/t/recommended-saves-and-loads-of-dataframe-jld-csv-etc/9074/9 "2020-08-30T03:36:22Z")

</div>

[JDF.jl](https://github.com/xiaodaigh/JDF.jl) works great if you don’t need interop with Python or R