# Converting NamedTuple to DataFrame seems expensive?

**URL:** https://discourse.julialang.org/t/converting-namedtuple-to-dataframe-seems-expensive/38674
**Category:** New to Julia
**Created:** [May 3, 2020, 2:30pm UTC](https://discourse.julialang.org/t/converting-namedtuple-to-dataframe-seems-expensive/38674 "2020-05-03T14:30:10Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)
#### Post date: [May 3, 2020, 2:30pm UTC](https://discourse.julialang.org/t/converting-namedtuple-to-dataframe-seems-expensive/38674/1 "2020-05-03T14:30:10Z")

</div>

I have a huge NamedTuple and

`DataFrame(larged_namedtuple)`

takes a long time like 12s and uses quite a bit of RAM

 ![image](https://global.discourse-cdn.com/julialang/original/3X/0/f/0f744b018c11a35f521b1b59f75fc2f66754c62c.png)

The eventual target for me is a DataFrame but I created the named tuple so that users can choose the sink they want. Is it better to not create the named tuple and just create the DataFrame? That would force a dependency on DataFrame on a package that I am only a potential contributor to, so if it can be avoided would be good.

Any good solutions?

---

<div class="post-metadata">

### Author: ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [May 3, 2020, 3:01pm UTC](https://discourse.julialang.org/t/converting-namedtuple-to-dataframe-seems-expensive/38674/2 "2020-05-03T15:01:09Z")

</div>

Use `copycols = false` in the `DataFrame` constructor.

If you can avoid a dependency on data frames that would be best.

---

<div class="post-metadata">

### Author: ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)
#### Post date: [May 3, 2020, 4:22pm UTC](https://discourse.julialang.org/t/converting-namedtuple-to-dataframe-seems-expensive/38674/3 "2020-05-03T16:22:19Z")

</div>

Huge in what sense? Many columns?

---

<div class="post-metadata">

### Author: ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)
#### Post date: [May 3, 2020, 4:29pm UTC](https://discourse.julialang.org/t/converting-namedtuple-to-dataframe-seems-expensive/38674/4 "2020-05-03T16:29:24Z")

</div>

100s col 26m rows

---

<div class="post-metadata">

### Author: ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)
#### Post date: [May 3, 2020, 4:30pm UTC](https://discourse.julialang.org/t/converting-namedtuple-to-dataframe-seems-expensive/38674/5 "2020-05-03T16:30:54Z")

</div>

> [@pdeffebach](#):
>
> Use `copycols = false` in the `DataFrame` constructor.

Does it make the dataframe immutable?

---

<div class="post-metadata">

### Author: ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [May 3, 2020, 4:38pm UTC](https://discourse.julialang.org/t/converting-namedtuple-to-dataframe-seems-expensive/38674/6 "2020-05-03T16:38:24Z")

</div>

As far as I can tell, no. I think the issue you are thinking of is `CSV.read` which used to return an immutable `AbstractArray` type that would cause problems with `copycols = false`.

---

<div class="post-metadata">

### Author: ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)
#### Post date: [May 3, 2020, 4:58pm UTC](https://discourse.julialang.org/t/converting-namedtuple-to-dataframe-seems-expensive/38674/7 "2020-05-03T16:58:01Z")

</div>

@xiaodai, it is a lot easier to help you if you provide more concrete information. Are you passing one named tuple with 100s fields and each field is a vector with one element per row? Or are you passing a vector of named tuples? Or an iterator of named tuples?

---

<div class="post-metadata">

### Author: ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)
#### Post date: [May 3, 2020, 11:44pm UTC](https://discourse.julialang.org/t/converting-namedtuple-to-dataframe-seems-expensive/38674/8 "2020-05-03T23:44:49Z")

</div>

> [@davidanthoff](#):
>
> is a lot easier to help you if you provide more concrete information

Firstly, I create 100s of vectors using the multi-threading, so I have a (unamed) tuple of 100s of materialized vectors.

Then I create names for them using namedtuple. Come to think of it, I can just create DataFrame from tuple. And then give them names so I can skip the name tuple dependency.
