Status of Julia database libraries

General Usage
144

Yes, you would use LibPQ.jl and I believe others think it’s now good (I also see recent activity on it). I used it minimally a few years back, and it got the job done.

Clicking on the other link showed also SQLREPL.jl (which is great, I believe I nerdsniped that package into existence) and https://www.algebraicjulia.org which might be of interest (I’ve not taken a good look at it to see how good/useful). Both, and more packages useful with Postgres/LibPQ and other databases.

If you ever want to change databases you might consider ODBC.jl, but if not just go with LibPQ, and I think doing so might also be faster. There are not many reasons to abandon PostgreSQL, and even if you do you could use LibPQ with some alternative databases.

Note, those others are archived at GitHub. You see such if you click their links (and I suppose GitHub also has an API for this that Pkg could plug into), but you do not see such by simply installing packages from Julia. Packages are never removed from the General registry, blocking installation (for security reasons), and you see at the source (on GitHub) if no longer maintained, sometimes in the README and/or a banner. I’ve never heard of a package changing to warning at runtime (or install time).

For a while the important package LoopVectorization.jl was unmaintained, and its speed-up code didn’t work for 1.11 (it might work again now, didn’t confirm), so you could still install it, but with the optimizations there disabled. The README mentioned this, and I thought a warning was emitted at load time (it seems I was wrong; for unmaintained packages maybe Pkg, could and should let you know when insttaling, if not when loading such packages):

145

Thank you for your feedback.

Are the “PostgreSQL” and “Postgres” packages deprecated? I think this is what “archived” implies?

146

Yes, sort of. Deprecated means you shouldn’t use (that API) since it is planned to be dropped in later version. Here no later versions are planned. It’s perfectly fine to use archived projects (in general), e.g. if you still have code using them (and they are bug-free). If such projects need to be modified, then anyone can fork or project and that way take over, but here no need, LibPQ is preferred.

LibPQ uses C code, i.e. in libPQ_jll, i.e. the official PostgreSQL driver, which is very fast. I think at least one of the other is pure Julia (needs not be a goal, it’s great to reuse C code), and both of those other are clearly outdated (probably do not install in latest Julia 1.0+.

That’s probably not something to worry about. I see LibPQ conforms to DBInterface, and I suppose that’s to easily allow moving to a driver for a different database. ODBC might have a speed-penalty, I’m not sure (I believe it was a worry historically before Julia, not sure by now), at least I’m very unsure about recommending using it.

@anon91151494, I’m not a scientist, it’s hard to generalize, likely many use SQL with their language and/or other things like HDF5.jl, Arrow.jl, Parquet2.jl, and even CSV (or JSON) if still commonly used. Sometimes SQL (and relational databases) are avoided for some very good reasons using such file formats. Most alternative databases to relational have though been integrated into PostgreSQL (or at least as extensions. There’s a very good video on SQL not being the future though, with an alternative API based on ranges. Found it! (it’s not simply the noSQL movement, such like Mongo users, have mostly gone back to using PostgreSQL).

For (SQL) databases you might want to use TidierDB.jl (e.g. with LibPQ backend).

Some data, e.g. for particle physics, certainly does not fit relational databases (and SQL) well:

And this one is great GitHub - JuliaHEP/AwkwardArray.jl: Awkward Array in Julia mirrors the Python library, enabling effortless zero-copy data exchange between Julia and Python!

3 Likes
147

Thanks - that’s great