# How to transform DataFrame back to its data source

**URL:** <https://discourse.julialang.org/t/how-to-transform-dataframe-back-to-its-data-source/90536>\
**Category:** General Usage\
**Tags:** dataframes\
**Created:** [November 20, 2022, 12:52pm UTC](https://discourse.julialang.org/t/how-to-transform-dataframe-back-to-its-data-source/90536 "2022-11-20T12:52:56Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![alexgr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alexgr/32/34616_2.png) [@alexgr](https://discourse.julialang.org/u/alexgr)\
**Post date:** [November 20, 2022, 12:52pm UTC](https://discourse.julialang.org/t/how-to-transform-dataframe-back-to-its-data-source/90536/1 "2022-11-20T12:52:56Z")

</div>

How would you approach going back from DataFrame to a vector of Records given this code?

```julia
struct Record 
    A::UInt32
    B::Float32
    C::Float32
end

# get bytes from stream; Vector{UInt8}
buffer = rand(UInt8, sizeof(Record) * 5) 

# reinterpret bytes as vector of Records 
records = reinterpret(Record, buffer) 

# build dataframe
df = DataFrame(records)

# how to get back from dataframe to a vector of Records?

```

---

<div class="post-metadata">

**Author:** ![bertschi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bertschi/32/33462_2.png) [@bertschi](https://discourse.julialang.org/u/bertschi)\
**Post date:** [November 20, 2022, 1:38pm UTC](https://discourse.julialang.org/t/how-to-transform-dataframe-back-to-its-data-source/90536/2 "2022-11-20T13:38:21Z")

</div>

The following will work:

```julia
[Record(x...) for x in eachrow(df)]

```

It is a bit dangerous though as the data frame columns have to be in the correct order, i.e., the following is not what you want, but might or might not work depending on types:

```julia
[Record(x...) for x in eachrow(select(df, :B, :A, :C))]

```

The best way would probably to define a `Record` constructor for reconstructing from data frame rows:

```julia
Record(row::DataFrameRow) = Record(row.A, row.B, row.C)

Record.(eachrow(select(df, :B, :A, :C))) # does the right thing even with permuted columns

```

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [November 20, 2022, 4:04pm UTC](https://discourse.julialang.org/t/how-to-transform-dataframe-back-to-its-data-source/90536/3 "2022-11-20T16:04:56Z")

</div>

> [@bertschi](#):
>
> The best way would probably to define a `Record` constructor for reconstructing from data frame rows:

This means you need to add the whole heavy DataFrames dependency to where you define `Record`.  
A cleaner, more general, and no-deps approach is to define a kwargs constructor:

```julia
@kwdef struct Record 
   A::UInt32
   B::Float32
   C::Float32
end

```

Then,

```julia
[Record(; x...) for x in eachrow(df)]

```

works no matter the column order.

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [November 20, 2022, 8:31pm UTC](https://discourse.julialang.org/t/how-to-transform-dataframe-back-to-its-data-source/90536/4 "2022-11-20T20:31:34Z")

</div>

```julia
using Tables
reinterpret(Record, rowtable(df))

```

---

<div class="post-metadata">

**Author:** ![bertschi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bertschi/32/33462_2.png) [@bertschi](https://discourse.julialang.org/u/bertschi)\
**Post date:** [November 20, 2022, 8:36pm UTC](https://discourse.julialang.org/t/how-to-transform-dataframe-back-to-its-data-source/90536/5 "2022-11-20T20:36:29Z")

</div>

[quote="aplavin, post:3, topic:90536]  
This means you need to add the whole heavy DataFrames dependency to where you define `Record`.  
[/quote]  
Not necessarily, as you can define the method later and elsewhere, i.e., in the code that is working with data frames already and requires that functionality.  
Would probably also prefer to define constructors close to the definition of a struct, but Julia allows other options here as well.  
[quote="aplavin, post:3, topic:90536]  
A cleaner, more general, and no-deps approach is to define a kwargs constructor.  
[/quote]  
Agreed, also more general as named tuples are more widespread and not tied to data frames.

---

<div class="post-metadata">

**Author:** ![alexgr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alexgr/32/34616_2.png) [@alexgr](https://discourse.julialang.org/u/alexgr)\
**Post date:** [November 22, 2022, 8:53pm UTC](https://discourse.julialang.org/t/how-to-transform-dataframe-back-to-its-data-source/90536/6 "2022-11-22T20:53:15Z")

</div>

Thank you all. I learned something from all of you.

Since column order is not a concern in my use case, I ended using @rocco_sprmnt21’s solution because it is much more performant.

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [November 22, 2022, 10:20pm UTC](https://discourse.julialang.org/t/how-to-transform-dataframe-back-to-its-data-source/90536/7 "2022-11-22T22:20:34Z")

</div>

> [@bertschi](#):
>
> ```julia
> Record(row::DataFrameRow) = Record(row.A, row.B, row.C)
> 
> Record.(eachrow(select(df, :B, :A, :C)))
> 
> ```

Couldn’t we write your second line simply like this:

```julia
Record.(eachrow(df))

```

This would then be faster than the other solutions posted so far.

---

<div class="post-metadata">

**Author:** ![bertschi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bertschi/32/33462_2.png) [@bertschi](https://discourse.julialang.org/u/bertschi)\
**Post date:** [November 22, 2022, 11:36pm UTC](https://discourse.julialang.org/t/how-to-transform-dataframe-back-to-its-data-source/90536/8 "2022-11-22T23:36:47Z")

</div>

Sure, just wanted to show that the reconstruction is correct even when you permute the columns.