# Best Data Manipulation packages 2020-09 \[video\]

**URL:** <https://discourse.julialang.org/t/best-data-manipulation-packages-2020-09-video/46690>\
**Category:** Data\
**Created:** [September 16, 2020, 6:14am UTC](https://discourse.julialang.org/t/best-data-manipulation-packages-2020-09-video/46690 "2020-09-16T06:14:08Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [September 16, 2020, 6:14am UTC](https://discourse.julialang.org/t/best-data-manipulation-packages-2020-09-video/46690/1 "2020-09-16T06:14:08Z")

</div>

https://www.youtube.com/embed/q_P2H_ZXVxI?feature=oembed&wmode=opaque&list=PLPQ4FAXPujHMhG1IcgHkiwumz5VqecvG2&start=0

According to me, basically this. Obvious, they can’t deal with larger-than-RAM data but that’s another story.

DataFrames.jl ([GitHub - JuliaData/DataFrames.jl: In-memory tabular data in Julia](https://github.com/JuliaData/DataFrames.jl))

DataFramesMeta.jl ([GitHub - JuliaData/DataFramesMeta.jl: Metaprogramming tools for DataFrames](https://github.com/JuliaData/DataFramesMeta.jl))

DataConvenience.jl ([GitHub - xiaodaigh/DataConvenience.jl: Convenience functions missing in Julia](https://github.com/xiaodaigh/DataConvenience.jl))

Pipe.jl ([https://github.com/oxinabox/Pipe.jl](https://github.com/oxinabox/Pipe.jl))

Lazy.jl ([https://github.com/MikeInnes/Lazy.jl](https://github.com/MikeInnes/Lazy.jl))

---

<div class="post-metadata">

**Author:** ![Jakob](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@Jakob](https://discourse.julialang.org/u/Jakob)\
**Post date:** [September 16, 2020, 5:12pm UTC](https://discourse.julialang.org/t/best-data-manipulation-packages-2020-09-video/46690/2 "2020-09-16T17:12:36Z")

</div>

I usually just roll with DataFrames.jl + Pipe.jl for these kinds of things. Could you briefly explain what kind of functionality the additional packages provide that you’re missing from those two?

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [September 17, 2020, 1:21am UTC](https://discourse.julialang.org/t/best-data-manipulation-packages-2020-09-video/46690/3 "2020-09-17T01:21:12Z")

</div>

> [@Jakob](#):
>
> Pipe.jl

I mentioned Pipe.jl as I like Lazy’s better

```julia
@> df begin
   group(:grp)
   combine(:col1=>mean=>:mean_col1
end

```

vs

```julia
@pipe df |>
   group(_, :grp) |>
   combine(_, :col1=>mean=>:mean_col1)
end

```

but there is more typing.

But `using Lazy` is dangerous a it exports `groupby` which clashes with `DataFrames.groupby`.

So `using DataConvenience` is what I prefer as it only (re)exports `@>`. Pluls, it h as other convenience functions I like, like sampling a dataframe with `sample(df, 0.05)`.

DataFramesMeta.jl can do things like

`@transform(df, x = fn(:y))` instead of `transform(df, :y => fn => :x)`

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [January 16, 2021, 1:43pm UTC](https://discourse.julialang.org/t/best-data-manipulation-packages-2020-09-video/46690/4 "2021-01-16T13:43:28Z")

</div>

BTW Lazy.jl and Pipe.jl has been dropped and replaced by Chain.jl

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [May 19, 2021, 2:24pm UTC](https://discourse.julialang.org/t/best-data-manipulation-packages-2020-09-video/46690/5 "2021-05-19T14:24:11Z")

</div>

stuff here is outdated. See [[Video] Best Julia packages for manipulating tabular (dataframe) data](https://discourse.julialang.org/t/video-best-julia-packages-for-manipulating-tabular-dataframe-data/61451) for latest recommendations