# Trying to bench two code samples and I'm getting different results @time vs @btime

**URL:** <https://discourse.julialang.org/t/trying-to-bench-two-code-samples-and-im-getting-different-results-time-vs-btime/46576>\
**Category:** New to Julia\
**Created:** [September 14, 2020, 6:53am UTC](https://discourse.julialang.org/t/trying-to-bench-two-code-samples-and-im-getting-different-results-time-vs-btime/46576 "2020-09-14T06:53:40Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![conditionality](https://avatars.discourse-cdn.com/v4/letter/c/4af34b/32.png) [@conditionality](https://discourse.julialang.org/u/conditionality)\
**Post date:** [September 14, 2020, 6:53am UTC](https://discourse.julialang.org/t/trying-to-bench-two-code-samples-and-im-getting-different-results-time-vs-btime/46576/1 "2020-09-14T06:53:41Z")

</div>

I have a DataFrame “results” that is sorted by \_.date. I’m experimenting to see the difference between using a filter (A):  
`results |> @filter(_.date >= Dates.Date(2018,1,1) && _.date <= Dates.Date(2019,1,1))`  
vs direct indexing (B):

```julia
let idx_first = findfirst(dt -> dt == Dates.Date(2018,1,1), results.date)
       idx_last = findlast(dt -> dt == Dates.Date(2019,1,1), results.date)
       @view results[idx_first:idx_last, :]
end

```

When running @time I get (I’ve ran @time several times in a row, so no JIT involved):  
(A): 0.013382 seconds (5.74 k allocations: 336.941 KiB)  
(B): 172.499 μs (12 allocations: 416 bytes)  
Clearly (B) wins. But when I run @btime in front of the two code samples I get:  
(A): 60.500 μs (121 allocations: 8.31 KiB)  
(B): 0.040810 seconds (104.54 k allocations: 6.316 MiB)  
and (B) does a lot of allocations, thus (A) wins. Can someone explain this to me?

When I collect both into dataframes at the end  
`results |> @filter(_.date >= Dates.Date(2018,1,1) && _.date <= Dates.Date(2019,1,1)) |> DataFrame`  
vs

```julia
let idx_first = findfirst(dt -> dt == Dates.Date(2018,1,1), results.date)
       idx_last = findlast(dt -> dt == Dates.Date(2019,1,1), results.date)
       results[idx_first:idx_last, :] |> DataFrame
end

```

then (B) is the clear winner in both (@time vs @btime) cases.

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [September 14, 2020, 7:24am UTC](https://discourse.julialang.org/t/trying-to-bench-two-code-samples-and-im-getting-different-results-time-vs-btime/46576/2 "2020-09-14T07:24:45Z")

</div>

The result using `@time` is expected to be very noisy for short duration tasks, solving this problem is the whole purpose of `@btime`. I would thus not use `@time` at all and only use `@btime` for benchmarking tasks that takes less than a second or so.

---

<div class="post-metadata">

**Author:** ![tisztamo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tisztamo/32/16200_2.png) [@tisztamo](https://discourse.julialang.org/u/tisztamo)\
**Post date:** [September 14, 2020, 7:46am UTC](https://discourse.julialang.org/t/trying-to-bench-two-code-samples-and-im-getting-different-results-time-vs-btime/46576/3 "2020-09-14T07:46:37Z")

</div>

Yes, and `@time` may also measure compilation time. But there is a real discrepancy here, because `@btime` returns the minimum, so it should just win over `@time`.

---

<div class="post-metadata">

**Author:** ![tk3369](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tk3369/32/2824_2.png) [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Post date:** [September 16, 2020, 3:18am UTC](https://discourse.julialang.org/t/trying-to-bench-two-code-samples-and-im-getting-different-results-time-vs-btime/46576/4 "2020-09-16T03:18:50Z")

</div>

The `@benchmark` macro would give you more information such as min/max/mean of the trials, should that be desired.