# Reading Data Is Still Too Slow

**URL:** <https://discourse.julialang.org/t/reading-data-is-still-too-slow/17919>\
**Category:** Data\
**Created:** [November 23, 2018, 5:54pm UTC](https://discourse.julialang.org/t/reading-data-is-still-too-slow/17919 "2018-11-23T17:54:56Z")\
**Posts on this page:** 1\
**Showing post:** 26

<div class="post-metadata">

**Author:** ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Post date:** [November 24, 2018, 11:47pm UTC](https://discourse.julialang.org/t/reading-data-is-still-too-slow/17919/26 "2018-11-24T23:47:22Z")

</div>

I tried to load a feather file now that seems roughly similar to the CRSP file described [here](http://julia.cookbook.tips/doku.php?id=fileformats): 90 million rows, a mix of `Int` and `Float64` columns. The file is about 8.5 GB on disc.

With the current released version of [DataFrames.jl](https://github.com/JuliaData/DataFrames.jl), it takes about 80 seconds to load on my system (I do have a very fast system 🙂 ). When I use the `master` branch of [DataFrames.j](https://github.com/JuliaData/DataFrames.jl), it takes somewhere between 7-15 seconds to load. All of these numbers are for [FeatherFiles.jl](https://github.com/queryverse/FeatherFiles.jl).

So I suspect (or hope 😉 ) that @iwelch’s numbers from above are with the released [DataFrames.jl](https://github.com/JuliaData/DataFrames.jl), in which case we might actually have something very competitive once we get a new [DataFrames.jl](https://github.com/JuliaData/DataFrames.jl) release out.

Caveat is that I haven’t tried a column with missing values yet.

---

_[View the full topic](https://discourse.julialang.org/t/reading-data-is-still-too-slow/17919)._