I have a very large database A, and a much smaller one B. Since A is so large, I cannot load it in memory at once. I keep A as a gzip compressed CSV file on disk, and I want to load it lazily. On the other hand B is small and fits well into memory. The goal is to perform a join between them, on a c…

Lazy database join

Jakob April 10, 2020, 6:45am 4

This is possible with JuliaDB, see the docs. I’ve had some problems in the past with these joins and I’m not sure if JuliaDB is being actively maintained at the moment but often it also just worked fine so I’d just give it a try.

Topic		Replies	Views
What's the best way to work with millions of rows of data? Performance	7	2037	February 24, 2020
The state of DataFrames.jl H2O benchmark Package Announcements dataframes	53	9114	January 1, 2025
JuliaDB, tutorial with large datasets and other questions General Usage tutorials	0	824	January 20, 2020
Larger than memory table format with lazy reads? General Usage array , dataframes , juliadb	5	1166	August 19, 2020
[ANN] JDBC.jl Rejuvenated Data announcement , dataframes	3	1212	March 20, 2018

Lazy database join

Related topics