# First part benchmarking medium-data tools

**URL:** <https://discourse.julialang.org/t/first-part-benchmarking-medium-data-tools/28260>\
**Category:** Data\
**Tags:** juliadb\
**Created:** [September 1, 2019, 12:06pm UTC](https://discourse.julialang.org/t/first-part-benchmarking-medium-data-tools/28260 "2019-09-01T12:06:11Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [September 1, 2019, 12:06pm UTC](https://discourse.julialang.org/t/first-part-benchmarking-medium-data-tools/28260/1 "2019-09-01T12:06:11Z")

</div>

I compared and contrast JuliaDB with Dask and disk.frame.

[http://diskframe.com/articles/vs-dask-juliadb.html](http://diskframe.com/articles/vs-dask-juliadb.html)

Probably more to come, but JuliaDB needs to mature before it can be a contender in these benchmarks.

---

<div class="post-metadata">

**Author:** ![mwolff](https://avatars.discourse-cdn.com/v4/letter/m/c2a13f/32.png) [@mwolff](https://discourse.julialang.org/u/mwolff)\
**Post date:** [September 1, 2019, 12:29pm UTC](https://discourse.julialang.org/t/first-part-benchmarking-medium-data-tools/28260/2 "2019-09-01T12:29:33Z")

</div>

I don’t know much about JuliaDB but could it be that there are some precompile times from Julia in these benchmark tests that slows down the result?

---

<div class="post-metadata">

**Author:** ![dilumaluthge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dilumaluthge/32/29283_2.png) [@dilumaluthge](https://discourse.julialang.org/u/dilumaluthge)\
**Post date:** [September 1, 2019, 12:33pm UTC](https://discourse.julialang.org/t/first-part-benchmarking-medium-data-tools/28260/3 "2019-09-01T12:33:29Z")

</div>

Instead of using the `@time` macro, you should use the `@btime` macro from the BenchmarkTools package.

In general, you should use the BenchmarkTools package for benchmarking.

---

<div class="post-metadata">

**Author:** ![tlienart](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tlienart/32/7640_2.png) [@tlienart](https://discourse.julialang.org/u/tlienart)\
**Post date:** [September 1, 2019, 4:20pm UTC](https://discourse.julialang.org/t/first-part-benchmarking-medium-data-tools/28260/4 "2019-09-01T16:20:17Z")

</div>

given how much benchmarking @xiaodai has done for csv reading etc, I imagine he’s well aware of this 🙂 . In my (limited) personal experience, JDB is awesome in principle but I had the same impressions than the ones that are summarised in the post and was always better off using DataFrames though that may just reflect the nature of the data I was playing with.

As an aside, I think as a community we should be wary of only just telling people to look at @btime because it looks better; while it’s certainly relevant in most cases where functions are used multiple times etc; for things like loading files and transforming to a given format, (new) users who would do such operations once or twice in a session would very much experience the compilation time and be disappointed that their experience doesn’t match promised speedy results. But then that’s the TTFP issue which has already been discussed at length…

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [September 1, 2019, 9:41pm UTC](https://discourse.julialang.org/t/first-part-benchmarking-medium-data-tools/28260/5 "2019-09-01T21:41:51Z")

</div>

> [@tlienart](#):
>
> while it’s certainly relevant in most cases where functions are used multiple times etc; for things like loading files and transforming to a given format, (new) users who would do such operations once or twice in a session would very much experience the compilation time and be disappointed that their experience doesn’t match promised speedy results.

Couldn’t have put it better myself