# A serious data start-up structured around a Julia data manipulation framework for larger-than-RAM data

**URL:** <https://discourse.julialang.org/t/a-serious-data-start-up-structured-around-a-julia-data-manipulation-framework-for-larger-than-ram-data/119430>\
**Category:** Offtopic\
**Created:** [September 15, 2024, 12:13pm UTC](https://discourse.julialang.org/t/a-serious-data-start-up-structured-around-a-julia-data-manipulation-framework-for-larger-than-ram-data/119430 "2024-09-15T12:13:52Z")\
**Posts on this page:** 1\
**Showing post:** 23

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [September 16, 2024, 6:07pm UTC](https://discourse.julialang.org/t/a-serious-data-start-up-structured-around-a-julia-data-manipulation-framework-for-larger-than-ram-data/119430/23 "2024-09-16T18:07:48Z")

</div>

See also [[ANN] SQLCollections.jl – use Julia data manipulation functions for databases](https://discourse.julialang.org/t/ann-sqlcollections-jl-use-julia-data-manipulation-functions-for-databases/119244) for a somewhat complementary direction. SQLCollections allows using regular Julia operations on SQL databases, nice for big/out of memory/remote data.  
Should be fully composable!  
Some features are possible but not implemented yet (it’s just ~200 LOC) – such as joins or using arbitrary Julia UDFs.

It would be nice to understand and list specific limitations of such an approach of using SQL databases compared to querying pure-Julia structures, as @xiaodai plans. Some of them can realistically be possible to overcome indeed!

---

_[View the full topic](https://discourse.julialang.org/t/a-serious-data-start-up-structured-around-a-julia-data-manipulation-framework-for-larger-than-ram-data/119430)._