# DataFrames Complexity

**URL:** <https://discourse.julialang.org/t/dataframes-complexity/47069>\
**Category:** Data\
**Tags:** dataframes\
**Created:** [September 22, 2020, 1:25pm UTC](https://discourse.julialang.org/t/dataframes-complexity/47069 "2020-09-22T13:25:42Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![NafQan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nafqan/32/15606_2.png) [@NafQan](https://discourse.julialang.org/u/NafQan)\
**Post date:** [September 22, 2020, 1:25pm UTC](https://discourse.julialang.org/t/dataframes-complexity/47069/1 "2020-09-22T13:25:43Z")

</div>

Sorry for being blunt, but a question that I am having for a while now and is in someway bothering me:  
Is there anything that DataFrames.jl does which can’t be done with arrays and matrices in a more efficient, less complicated way, and more readable code?  
Aside from giving us a slow array of type {Any} what is the real added value one gets from using DataFrames?

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [September 22, 2020, 1:27pm UTC](https://discourse.julialang.org/t/dataframes-complexity/47069/2 "2020-09-22T13:27:29Z")

</div>

I think there are a lot of things that are easier with data frames! It’s hard to comment on this without a specific example.

What do you mean by a slow array type `{Any}`? The vectors in DataFrames have specific eltypes, they are normal Julia vectors and carry all those performance benefits with them.

---

<div class="post-metadata">

**Author:** ![nickeubank](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nickeubank/32/3878_2.png) [@nickeubank](https://discourse.julialang.org/u/nickeubank)\
**Post date:** [September 22, 2020, 1:33pm UTC](https://discourse.julialang.org/t/dataframes-complexity/47069/3 "2020-09-22T13:33:25Z")

</div>

Working with heterogenous tables. I work almost entirely with datasets that are part string, part categorical, and part numeric, and without `DataFrames` it’d be near impossible to do this efficiently.

---

<div class="post-metadata">

**Author:** ![johnmyleswhite](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnmyleswhite/32/31_2.png) [@johnmyleswhite](https://discourse.julialang.org/u/johnmyleswhite)\
**Post date:** [September 22, 2020, 1:46pm UTC](https://discourse.julialang.org/t/dataframes-complexity/47069/4 "2020-09-22T13:46:02Z")

</div>

> [@NafQan](#):
>
> Is there anything that DataFrames.jl does which can’t be done with arrays and matrices in a more efficient, less complicated way, and more readable code?

Maybe it’s helpful to flip your question: can you provide any example in which a `Matrix{Any}` is substantially more efficient than a DataFrame with one `Float64` column and one `String` column? You seem to be assuming you almost always can use `Matrix{Any}`, but you haven’t given any examples yet.

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [September 22, 2020, 1:52pm UTC](https://discourse.julialang.org/t/dataframes-complexity/47069/5 "2020-09-22T13:52:07Z")

</div>

```julia
@pipe df |>
  group_by(_, :grp) |>
  combine(_, :col1 => mean => :mean_col1)

```

is pretty readable to me. How do you do that with a `Matrix`?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [September 23, 2020, 9:06am UTC](https://discourse.julialang.org/t/dataframes-complexity/47069/6 "2020-09-23T09:06:18Z")

</div>

> [@NafQan](#):
>
> Is there anything that DataFrames.jl does which can’t be done with arrays and matrices in a more efficient, less complicated way, and more readable code?

For one thing, keeping track of column indexes instead of names would make almost all code less readable, and more brittle.

> [@NafQan](#):
>
> Sorry for being blunt

Being blunt is acceptable (if not ideal), but please do a bit of research before asking a question like this. No, DataFrames are not a

> [@NafQan](#):
>
> a slow array of type {Any}