# Help improving the speed of a DataFrames operation

**URL:** https://discourse.julialang.org/t/help-improving-the-speed-of-a-dataframes-operation/107615
**Category:** Performance
**Tags:** performance, query, dataframes
**Created:** [December 14, 2023, 3:20pm UTC](https://discourse.julialang.org/t/help-improving-the-speed-of-a-dataframes-operation/107615 "2023-12-14T15:20:21Z")
**Posts on this page:** 1
**Showing post:** 6

<div class="post-metadata">

### Author: ![abelsiqueira](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abelsiqueira/32/47269_2.png) [@abelsiqueira](https://discourse.julialang.org/u/abelsiqueira)
#### Post date: [December 15, 2023, 3:18pm UTC](https://discourse.julialang.org/t/help-improving-the-speed-of-a-dataframes-operation/107615/6 "2023-12-15T15:18:32Z")

</div>

The code is defined inside a function, say `main`, that I have not written here.  
The `df_flows` variable is defined inside that function.  
Instead of writing functions, I am using the explicit access to the variable `df_flows.rp`. Since `df_flows` is a local variable, I think that using a function is not necessary here, right? I was just trying to make the example readable.

About the use case, it is a left join that I then group. I can try to explain below what I want:

Explanation in join terms:

- `leftjoin(df_cons, df_flows, on = [:rp, :asset => :to])`
- Compute the intersection of the `time_block` of left and right
- Multiply the resulting value by the `flow` column
- Sum `flow` by grouping by `df_cons`’ index

Per row explanation:

- For each row of `df_cons`
- Select/filter `df_flows` by matching `rp = row.rp` and `to = row.asset`
- Compute the intersection of the time blocks
- Multiply the resulting value by the `flow` column
- Sum `flow` and return

My current solution is to not do any use of DataFrames, and just use Dictionaries to store the indices of the non-zero flows. It is slow, but around 10x faster that this version. The full context is [Speeding up JuMP model creation with sets that depend on other indexes - #6 by slwu89](https://discourse.julialang.org/t/speeding-up-jump-model-creation-with-sets-that-depend-on-other-indexes/107333/6)

---

_[View the full topic](https://discourse.julialang.org/t/help-improving-the-speed-of-a-dataframes-operation/107615)._