# Passing DataFrame via ZMQ

**URL:** <https://discourse.julialang.org/t/passing-dataframe-via-zmq/12223>\
**Category:** General Usage\
**Created:** [July 6, 2018, 8:58pm UTC](https://discourse.julialang.org/t/passing-dataframe-via-zmq/12223 "2018-07-06T20:58:33Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![tk3369](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tk3369/32/2824_2.png) [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Post date:** [July 6, 2018, 8:58pm UTC](https://discourse.julialang.org/t/passing-dataframe-via-zmq/12223/1 "2018-07-06T20:58:33Z")

</div>

I want to pass a DataFrame via ZMQ. What would be the most effective way to do that?

I was thinking about serializing with JSON but not sure how to deserialize at the other end without assuming the internal structure of DataFrame, plus I lose the column type information… Any suggestions?

```julia
julia> y = JSON.json(DataFrame(a=1:10, b=rand(10), c=["abc$i" for i in 1:10]))
"{\"columns\":[[1,2,3,4,5,6,7,8,9,10],[0.33594662097541916,0.2867361571846303,0.31531071798857035,0.06042841515321973,0.10010429483452854,5.7254177610932544e-5,0.3638153944324909,0.3119557802963746,0.6338407676412947,0.7744933397995217],[\"abc1\",\"abc2\",\"abc3\",\"abc4\",\"abc5\",\"abc6\",\"abc7\",\"abc8\",\"abc9\",\"abc10\"]],\"colindex\":{\"lookup\":{\"a\":1,\"b\":2,\"c\":3},\"names\":[\"a\",\"b\",\"c\"]}}"

julia> x = JSON.parse(y)
Dict{String,Any} with 2 entries:
  "colindex" => Dict{String,Any}(Pair{String,Any}("names", Any["a", "b", "c"]),Pair{String,Any}("lookup", Dict{…
  "columns" => Any[Any[1, 2, 3, 4, 5, 6, 7, 8, 9, 10], Any[0.335947, 0.286736, 0.315311, 0.0604284, 0.100104, …

julia> z = DataFrame(x["columns"], Symbol.(x["colindex"]["names"]))
10×3 DataFrames.DataFrame
│ Row │ a │ b │ c │
├─────┼────┼────────────┼───────┤
│ 1 │ 1 │ 0.335947 │ abc1 │
│ 2 │ 2 │ 0.286736 │ abc2 │
│ 3 │ 3 │ 0.315311 │ abc3 │
│ 4 │ 4 │ 0.0604284 │ abc4 │
│ 5 │ 5 │ 0.100104 │ abc5 │
│ 6 │ 6 │ 5.72542e-5 │ abc6 │
│ 7 │ 7 │ 0.363815 │ abc7 │
│ 8 │ 8 │ 0.311956 │ abc8 │
│ 9 │ 9 │ 0.633841 │ abc9 │
│ 10 │ 10 │ 0.774493 │ abc10 │

julia> showcols(z)
10×3 DataFrames.DataFrame
│ Col # │ Name │ Eltype │ Missing │ Values │
├───────┼──────┼────────┼─────────┼───────────────────────┤
│ 1 │ a │ Any │ 0 │ 1 … 10 │
│ 2 │ b │ Any │ 0 │ 0.335947 … 0.774493 │
│ 3 │ c │ Any │ 0 │ abc1 … abc10 │

```

---

<div class="post-metadata">

**Author:** ![tk3369](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tk3369/32/2824_2.png) [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Post date:** [July 6, 2018, 10:03pm UTC](https://discourse.julialang.org/t/passing-dataframe-via-zmq/12223/2 "2018-07-06T22:03:15Z")

</div>

Looks like `Base.serialize` / `Base.deserialize` is exactly what I was looking for. I should have RTFM 🙂

Any other ideas welcome.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [July 7, 2018, 2:24pm UTC](https://discourse.julialang.org/t/passing-dataframe-via-zmq/12223/3 "2018-07-07T14:24:33Z")

</div>

If this is only passing between processes, and never saved on disk or to a database, then serialize/deserialize should be fine.

---

<div class="post-metadata">

**Author:** ![tk3369](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tk3369/32/2824_2.png) [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Post date:** [July 7, 2018, 5:51pm UTC](https://discourse.julialang.org/t/passing-dataframe-via-zmq/12223/4 "2018-07-07T17:51:24Z")

</div>

Yes, the data is transient for IPC only. Thanks, Scott.