# Hand rolled filter faster than filter function for dataframe?
**URL:** https://discourse.julialang.org/t/hand-rolled-filter-faster-than-filter-function-for-dataframe/27358
**Category:** General Usage
**Created:** [August 9, 2019, 7:23pm UTC](https://discourse.julialang.org/t/hand-rolled-filter-faster-than-filter-function-for-dataframe/27358 "2019-08-09T19:23:59Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@purplishrock](https://discourse.julialang.org/u/purplishrock)
#### Post date: [August 9, 2019, 7:23pm UTC](https://discourse.julialang.org/t/hand-rolled-filter-faster-than-filter-function-for-dataframe/27358/1 "2019-08-09T19:23:59Z")
From the MWE:
0.019227 seconds (313 allocations: 3.065 MiB)
0.037725 seconds (599.53 k allocations: 15.439 MiB, 8.73% gc time)
Faster and much, much fewer allocations. This result is better than what I see in the actual program where the hand-rolled filter (which doesn’t even pre-allocate for the result) is 4x faster. However my dataset is smaller, so there’s probably some 1-time overhead in the filter function that dominates. I think the MWE is a more realistic result.
What’s really interesting is if you replace my loop with something like
```julia
class = df[!,:b] .> 5
```
The allocations go way up and the run time is much worse:
0.273326 seconds (277.86 k allocations: 17.071 MiB)
0.044197 seconds (599.55 k allocations: 15.438 MiB, 9.76% gc time)
type instability ?
```julia
using DataFrames
function test()
n = 200000
df = DataFrame([[rand(1:10) for _ in 1:n],
[rand(1:10) for _ in 1:n],
[rand(1:10) for _ in 1:n]], [:a, :b, :c])
class = zeros(Bool, size(df,1))
m = 0
for i=1:size(df,1)
class[i] = df[i,:b] > 5
if class[i]
m += 1
end
end
println(m)
df1_ = @time df[class,:]
df1 = @time filter(r->r[:b] > 5, df)
end
test()
```
---
### Author:  [@quinnj](https://discourse.julialang.org/u/quinnj)
#### Post date: [August 9, 2019, 8:15pm UTC](https://discourse.julialang.org/t/hand-rolled-filter-faster-than-filter-function-for-dataframe/27358/2 "2019-08-09T20:15:17Z")
It doesn’t seem like you’re really timing the same things here:
```julia
df1_ = @time df[class, :]
```
This is just timing how long it takes to index a DataFrame w/ a Bool vector
vs.
```julia
df1 = @time filter(r->r[:b] > 5, df)
```
this is applying the `r->r[:b] > 5` function to each DataFrameRow and only returning the row when it’s true.
---
### Author:  [@purplishrock](https://discourse.julialang.org/u/purplishrock)
#### Post date: [August 9, 2019, 8:22pm UTC](https://discourse.julialang.org/t/hand-rolled-filter-faster-than-filter-function-for-dataframe/27358/3 "2019-08-09T20:22:43Z")
> [@quinnj](#):
>
> It doesn’t seem like you’re really timing the same things here:
You’re being very picky 😉
LOL. so, someday i’m going to ask a question on this forum that doesn’t embarrass me. But today is not that day.
Bad news - i’ve embarrassed myself again…
Good news - it shouldn’t be faster and it’s not.
Thank you!