# Disk based data manipulation framework needed

**URL:** <https://discourse.julialang.org/t/disk-based-data-manipulation-framework-needed/6303>\
**Category:** Data\
**Tags:** data\
**Created:** [October 7, 2017, 6:13am UTC](https://discourse.julialang.org/t/disk-based-data-manipulation-framework-needed/6303 "2017-10-07T06:13:21Z")\
**Posts on this page:** 3\
**Page:** 2

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [October 31, 2017, 2:15pm UTC](https://discourse.julialang.org/t/disk-based-data-manipulation-framework-needed/6303/21 "2017-10-31T14:15:18Z")

</div>

> [@ExpandingMan](#):
>
> I rarely have much need to store large amounts of data of custom types

Custom types are helpful especially when I have large amounts of data: I can compress the representation, and still work with it seamlessly. Eg using 16-bit integers for dates saves 75%, see

> **[GitHub - tpapp/FlexDates.jl: Dates with arbitrary integer types and epochs.](https://github.com/tpapp/FlexDates.jl)**
>
> Dates with arbitrary integer types and epochs. Contribute to tpapp/FlexDates.jl development by creating an account on GitHub.

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [October 31, 2017, 8:20pm UTC](https://discourse.julialang.org/t/disk-based-data-manipulation-framework-needed/6303/22 "2017-10-31T20:20:04Z")

</div>

i really like [fstpackage.org](http://fstpackage.org) it provides random access to rows and columns. The interface is in C++ so is not restricted to R

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [November 19, 2018, 5:46pm UTC](https://discourse.julialang.org/t/disk-based-data-manipulation-framework-needed/6303/23 "2018-11-19T17:46:18Z")

</div>

Split this topic on November 19, 2018, 5:46pm UTC

3 posts were split to a new topic: [Question: JuliaDB and regression models](https://discourse.julialang.org/t/question-juliadb-and-regression-models/17723)

[Previous page](https://discourse.julialang.org/t/disk-based-data-manipulation-framework-needed/6303.md?page=1)