# Database backed for DataFrames

**URL:** <https://discourse.julialang.org/t/database-backed-for-dataframes/94768>\
**Category:** Data\
**Tags:** dataframes, sql\
**Created:** [February 17, 2023, 3:24am UTC](https://discourse.julialang.org/t/database-backed-for-dataframes/94768 "2023-02-17T03:24:06Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![ayushpatnaikgit](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ayushpatnaikgit/32/25229_2.png) [@ayushpatnaikgit](https://discourse.julialang.org/u/ayushpatnaikgit)\
**Post date:** [February 17, 2023, 3:24am UTC](https://discourse.julialang.org/t/database-backed-for-dataframes/94768/1 "2023-02-17T03:24:07Z")

</div>

Any package in Julia like R’s [dbplyr](https://dbplyr.tidyverse.org/)?

dbplyr allows you to use remote database tables as if they are in-memory data frames by automatically converting dplyr code into SQL. I want to do the same with DataFrames and SQL.

---

<div class="post-metadata">

**Author:** ![mrufsvold](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mrufsvold/32/31600_2.png) [@mrufsvold](https://discourse.julialang.org/u/mrufsvold)\
**Post date:** [February 17, 2023, 4:36am UTC](https://discourse.julialang.org/t/database-backed-for-dataframes/94768/2 "2023-02-17T04:36:05Z")

</div>

[Query.jl](https://github.com/queryverse/Query.jl) says this is a goal in their readme, but it hasn’t been updated in a few years, and SQL gen doesn’t look like it’s been implemented.

It’s not an answer to your question, but [FunSQL jl](https://github.com/MechanicalRabbit/FunSQL.jl) has some pretty sweet functionality for generation SQL in native Julia.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [February 17, 2023, 9:05am UTC](https://discourse.julialang.org/t/database-backed-for-dataframes/94768/3 "2023-02-17T09:05:00Z")

</div>

A set of basic collection-like operations backed by an SQL database is available in [[ANN] SQLStore.jl: use SQL tables as persistent Julia collections](https://discourse.julialang.org/t/ann-sqlstore-jl-use-sql-tables-as-persistent-julia-collections/71796).

It presents tables with interface similar to arrays or dicts, see [the notebook](https://aplavin.github.io/SQLStore.jl/test/examples.html) for some examples. Stuff like `push!`, `filter`, `count`, etc.  
`SQLStore` is not a full Julia-SQL translation layer, so not exactly what you are looking for. Still, it is helpful for eg loading the required subset of the whole SQL table into memory. After that, just use familiar Julia in-memory tools (:

---

<div class="post-metadata">

**Author:** ![ayushpatnaikgit](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ayushpatnaikgit/32/25229_2.png) [@ayushpatnaikgit](https://discourse.julialang.org/u/ayushpatnaikgit)\
**Post date:** [February 18, 2023, 5:55am UTC](https://discourse.julialang.org/t/database-backed-for-dataframes/94768/4 "2023-02-18T05:55:47Z")

</div>

A bit ambitious, but I wonder if a package like dbplyr existed in Julia, then packages built on DataFrames, such as [Survey.jl](https://github.com/xKDR/Survey.jl) or [TSFrames.jl](https://github.com/xKDR/TSFrames.jl), could simply switch the backend to do everything out of memory.

Would something like this be interesting to build?

---

<div class="post-metadata">

**Author:** ![mrufsvold](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mrufsvold/32/31600_2.png) [@mrufsvold](https://discourse.julialang.org/u/mrufsvold)\
**Post date:** [February 20, 2023, 1:01am UTC](https://discourse.julialang.org/t/database-backed-for-dataframes/94768/5 "2023-02-20T01:01:51Z")

</div>

I would totally use that!

---

<div class="post-metadata">

**Author:** ![smishr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/smishr/32/46365_2.png) [@smishr](https://discourse.julialang.org/u/smishr)\
**Post date:** [February 21, 2023, 4:39am UTC](https://discourse.julialang.org/t/database-backed-for-dataframes/94768/6 "2023-02-21T04:39:05Z")

</div>

this would be very useful!

---

<div class="post-metadata">

**Author:** ![kdpsingh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kdpsingh/32/215018_2.png) [@kdpsingh](https://discourse.julialang.org/u/kdpsingh)\
**Post date:** [May 4, 2024, 3:05am UTC](https://discourse.julialang.org/t/database-backed-for-dataframes/94768/7 "2024-05-04T03:05:54Z")

</div>

A year late to the party but we have this now: [GitHub - TidierOrg/TidierDB.jl: Tidier database analysis in Julia, modeled after the dbplyr R package.](https://github.com/TidierOrg/TidierDB.jl)