# DataFrames and serialization

**URL:** <https://discourse.julialang.org/t/dataframes-and-serialization/26248>\
**Category:** General Usage\
**Created:** [July 11, 2019, 11:33am UTC](https://discourse.julialang.org/t/dataframes-and-serialization/26248 "2019-07-11T11:33:14Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![gdkrmr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdkrmr/32/2791_2.png) [@gdkrmr](https://discourse.julialang.org/u/gdkrmr)\
**Post date:** [July 11, 2019, 11:33am UTC](https://discourse.julialang.org/t/dataframes-and-serialization/26248/1 "2019-07-11T11:33:14Z")

</div>

I had to send a DataFrame (ca. 7000x1500, in my opinion not even that large) to several worker processes (ca. 40) and and the whole process was a pain:

```julia
@everywhere df = $df

```

took 3 minutes!

A colleague told me to write everything to disk and read it with the worker processes:

```julia
JLD.@save "df.jld" df

```

had to be interrupted because I lost patience

```julia
BSON.@save "df.bson" df
@everywhere BSON.@load "df.bson" df

```

saving finished but was very slow and reading the file into memory on the worker processes caused heavy swapping on the machine so that I had to kill all the processes to prevent the machine from freezing.

Finally I tried `JLD2` and it saved me:

```julia
JLD2.@save "df.jld2" df
@everywhere JLD2.@load "df.jld2" df

```

took 13s!

So what is everyone else doing wrong? I have seen some older issues about serializing `DataFrame`s which were mostly closed. And most importantly: How is it possible that this is faster over a (networked) filesystem than within the RAM of a single machine?

The same issue appeared with the return values of a `pmap`…