# ANN: Feather.jl v0.4.0 (lazy edition)

**URL:** <https://discourse.julialang.org/t/ann-feather-jl-v0-4-0-lazy-edition/14219>\
**Category:** Data\
**Created:** [August 28, 2018, 11:23pm UTC](https://discourse.julialang.org/t/ann-feather-jl-v0-4-0-lazy-edition/14219 "2018-08-28T23:23:03Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![ExpandingMan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/expandingman/32/866_2.png) [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Post date:** [August 28, 2018, 11:23pm UTC](https://discourse.julialang.org/t/ann-feather-jl-v0-4-0-lazy-edition/14219/1 "2018-08-28T23:23:03Z")

</div>

Hello all, it’s finally time to announce something that we’ve been working on and using extensively for a good while now, that is the new [Feather.jl](https://github.com/JuliaData/Feather.jl). (We still have to fix our docs, see [latest](http://juliadata.github.io/Feather.jl/latest/) for the v0.4.0 docs.)

For those of you who are not aware, feather is a lightweight binary tabular format. The most exciting thing about this new release is that it now supports completely lazy loading of tables via memory mapping. So, for example

```julia
df = Feather.read("a_table_that_is_3_GB.feather") # this happens almost instantly

# this *only* reads the first few rows, it doesn't matter how big your file is, this operation should take approximately the same amount of time
head(df)

# the lazy loading is completely seamless. if you perform an operation, *only* the data you are asking for gets touched
# the following *only* reads in data from column_A
mean(df.column_A) 

# the following *only* reads in rows 100 to 110 from column_B
mean(df.column_B[100:110])

# or *only* rows 100 through 110 of the entire dataframe
df[100:110, :]

# but you are still just working with ordinary DataFrames
typeof(df) == DataFrame

```

(Note that on Windows memory mapping is disabled by default, so you should do `Feather.read(filename, use_mmap=true)` to use memory mapping on Windows.)

One caveat is that the feather metadata does not yet support tables larger than 4 GB (ultimately these will involve multiple files). Fortunately, this should be relatively simple to implement, so if they don’t come up with a standard we’ll eventually have our own solution.

---

<div class="post-metadata">

**Author:** ![tk3369](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tk3369/32/2824_2.png) [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Post date:** [August 29, 2018, 5:03am UTC](https://discourse.julialang.org/t/ann-feather-jl-v0-4-0-lazy-edition/14219/2 "2018-08-29T05:03:15Z")

</div>

> [@ExpandingMan](#):
>
> Note that on Windows memory mapping is disabled by default

Good stuffs! Just curious why you decided to disable memory mapping on windows by default.

---

<div class="post-metadata">

**Author:** ![ExpandingMan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/expandingman/32/866_2.png) [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Post date:** [August 29, 2018, 1:23pm UTC](https://discourse.julialang.org/t/ann-feather-jl-v0-4-0-lazy-edition/14219/3 "2018-08-29T13:23:14Z")

</div>

> [@tk3369](#):
>
> Good stuffs! Just curious why you decided to disable memory mapping on windows by default.

See [here](https://github.com/JuliaData/Feather.jl/issues/84).

As memory mapping is now a central feature of feather I suspect this may have to be revisited at some point.