Correct way to transparently use SQLite.jl responses or (legacy) text files as a data source

[mkarikom]

I’ve really been enjoying SQLight.jl as a back end that just works. Is there any high level documentation on the relationship between SQLight.jl, Tables.jl, and DataFrames.jl?

Background: Unfortunately, the package I’m currently working on needs to also support text files as a data source for some users (in my area of scientific computing, databases are not widely used or understood), so DataFrames seems like the best representation since it can be populated by an SQLite response or a text file. However, I’m getting confused about the difference between Tables.jl and DataFrames.jl, since they seem to be very similar.

[dlakelan]

Tables.jl is an interface library, it defines generic functions, any type that implements those functions can be treated like a table source. This is true of DataFrames, but it’s also true of functions in the CSV library, the SQLite library, etc etc. It’s basically just ways to access data inside whatever types you are storing them.

[Tamas_Papp]

In addition to @dlakelan’s excellent answer, you may find the following post informative about motivation for Tables.jl:

Tables.jl: a table interface for everyone

I’ve had several people direct message me over the last couple of days asking for details about a new package I’ve been working: Tables.jl, so I thought I’d write up a quick post on what it is, where it came from, and where it’s going. There’s been talk for several years now of having an “AbstractTables.jl” package that would define the definitive common table interface for julia. Most efforts have died out pretty quickly or lost traction and were abandoned (I count at least 5 of these efforts …

[mkarikom]

Thanks, @dlakelan for the summary and @Tamas_Papp for the documentation. That provides all the info I need.