# Parallelize groupby combine

**URL:** <https://discourse.julialang.org/t/parallelize-groupby-combine/91710>\
**Category:** General Usage\
**Created:** [December 15, 2022, 8:03pm UTC](https://discourse.julialang.org/t/parallelize-groupby-combine/91710 "2022-12-15T20:03:48Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jar1](https://avatars.discourse-cdn.com/v4/letter/j/c0e974/32.png) [@jar1](https://discourse.julialang.org/u/jar1)\
**Post date:** [December 15, 2022, 8:03pm UTC](https://discourse.julialang.org/t/parallelize-groupby-combine/91710/1 "2022-12-15T20:03:48Z")

</div>

For `transform(df, cols => ByRow(f))` I can parallelize by replacing ByRow with a parallel `map` . Is it possible to parallelize `combine(groupby(df, cols), ...)` ?

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [December 15, 2022, 8:04pm UTC](https://discourse.julialang.org/t/parallelize-groupby-combine/91710/2 "2022-12-15T20:04:58Z")

</div>

see [Functions · DataFrames.jl](https://dataframes.juliadata.org/stable/lib/functions/#Multithreading-support)

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [December 15, 2022, 8:05pm UTC](https://discourse.julialang.org/t/parallelize-groupby-combine/91710/3 "2022-12-15T20:05:39Z")

</div>

There are the following things that could be parallelized in your question:

- `groupby`: it is parallelized
- parallelizing multiple operations in `...` - this is already done by default (you can disable it if you want)
- parallelize a single operation that is a custom function that produces one row per group - this is already done by default (you can disable it if you want)
- parallelize a single operation that is a custom function that produces many rows per group - currently parallelizing it is not supported (the reason is that composing an output of such an operation is hard to parallelize)
- parallelize a single operation that is a standard function that is optimized (like `mean`, `sum`) - this is currently not supported, we might add it, but we have not done so yet, because the custom aggregations we now have that use a single thread are fast and it was hard to find a good threshold when enabling multithreading gave benefits (for sure for tables having less than 1000000 rows the cost of spawning tasks was bigger than the benefit)