I tried updating it in various occasions but in all honesty, low-level C integration code is not my strength.
From a web developer’s perspective, PostgreSQL is arguably the most important RDBMS. For many years it was the only relational database offered by Heroku, for example.
Without a powerful, up to date PostgreSQL implementation, Julia can not be taken seriously for web development. It’s as simple as that.
Can we please do anything about it? We need a fast and modern implementation that goes beyond the hustle and the basic features of ODBC, that not only works, but also includes things like pub/sub, JSON types, etc.
1/ Is anybody looking for an OpenSource project to make a meaningful contribution? This would be a great fit - I’m happy to help with testing, documentation, etc.
2/ Can JuliaDB take this project and push it up the priority queue for an internship or a SoC project?
3/ Finally, if everything else fails, is anybody willing to do it for money? If so, message me, maybe we can gather some support in the Julia community and crowdfund the project and/or get some sponsors involved.
Any feedback on how to get this moving would be highly appreciated.
@ essenciary
Thank you for raising this point. Julia’s choice of databases has been rather dismal all along, but I was willing to accept this (an generally lacking web support) as part of working in a relatively new language.
However, lack of native support for Postgres would be a complete show stopper for the projects I’m working on.
I’m in a similar position to to the OP as I my knowledge of C/C++ is not enough to take over maintenance of the driver.
However, since data used for any purpose will come out of a DB and Postgres is arguable the best FOSS DB available the lack of support in Julia is particularly troubling.
Any help with this matter would be greatly appreciated!
I guess the amount of feedback this issue has generated is a direct indication of the community’s interest in Postgres
Sadly, the problem is not just Postgres.
One of my responses in Genie was to speed up integration with MySQL and have that as the default as of Julia v0.6. The MySQL.jl library is much better maintained and I want to thank the contributors for that.
Although Postgres has replaced MySQL as the default DB for the web in the last years, MySQL is still strong and it’s a viable alternative until the Postgres conundrum will hopefully resolve.
But like I said, it’s not that simple. It turned out that one of the MySQL.jl most important methods, mysql_execute is highly type unstable. For a method that can be called even tens of times during the duration of just one request - response cycle that is a very bad sign (issue opened here: https://github.com/JuliaDB/MySQL.jl/issues/84)
In my opinion, the Julia core team should treat these low-level libraries as “public utilities”. They are too important to be addressed by the market (and obviously that does not work).
I don’t see many other solutions right now. For Postgres, my first thought was to get some company to sponsor the library and hire a C developer to do it right. But of course, if C developers can be found, C developers that also know Julia are a rare bird. So it looks like a vicious cycle: without libraries, developers won’t come; without developers, libraries won’t be developed. Hence the need for the Julia core team to take responsibility for these low-level libraries to help the language grow.
And? The list of packages which look essential to certain parts of the usage spectrum is not very short. Even if the so-called julia core team decides to do priorisation and spend own effort on packages as intrastructure, still some time will pass.
I don’t think you really need a C developer to get the package to a usable state. Most of the work should happen in Julia code. Some knowledge of C is only needed to wrap the C API, which is much less demanding than writing C code.
Also, a lot of the work probably just involves porting the package to the new Julia versions, which should not require dealing with the C wrappers (or only marginally). Fixing deprecations is typically an area where new contributors can help, as the changes are generally self-contained. Help is always welcome, as the core Julia team is already busy with stabilizing the language for 1.0.
And… that would be a big step ahead. It’s a matter of principle - having up-to-date, fast, feature-rich core libraries would be a major point of sale for the language, even if it takes a bit of time.
It may be that the list is long, but I think many would agree that access to major databases should be high on top of the list.
Like I said, it’s not only deprecations. In the specific case of Postgres, even when it worked, the library lacked features. And then, how long can we go with just patching things? Until Postgres 10?
Sure, we can all push a broken car downhill - but that’s hardly a strategy.
Yet the first step to get the package working again will necessarily be to fix deprecations and port it to Julia 0.6. You offered your help in the OP, so all I’m saying is that this should be doable and useful way of spending your efforts.
As in all open source projects, people have their own priorities and you have very few ways to make them work on yours. The best (and maybe only) way to help making progress in an area that interests you is to contribute, or to pay somebody to contribute for you.
I have some experience with v0.6, wrapping C codes and use DBMS regularly; would be interested in pitching in towards the maintenance of PostgreSQL.jl. If anyone’s interested in collaborating let me know
I’m ready to handle testing, both manually and automated, I’ve used the library extensively. Happy to try it out in my projects and update / write more tests.
I already see a bit of activity in the issue referenced above. We can continue there.
To get it working on 0.6 (at least for my needs) it was sufficient to just merge some pending pull requests to PostgreSQL and DBI and run femtocleaner to clean up deprecation warnings. I have forks of PostgreSQL and DBI if you wanted to Pkg.clone them. I make no guarantees about their suitability or stability though! (and should probably point out that my needs are pretty minimal - just running queries and fetching their results)
Sadly, there aren’t many other features besides running queries, so that should cover most of the library. I would’ve loved to see pubsub support and other “exotic” features but I’m not holding my breath.
I have logged a new issue on the original PostgreSQL.jl repository. I could not see an option to file it on your fork. If you have time, can you look into it please?
I see the issue but unfortunately I don’t know anything about what an OID is or what kind of conversion you’d expect from it. If you can put together a minimal working example with a small sample database and query (or point me in the direction of a public database that you can reproduce the error on), we might be able to narrow down the problem.
