What is the current state of packages in Julia for accessing S3 object stores. In particular CEPH.
Are people just using boto with PyCall?
Forgive me if I have asked this one before. Goldfish bowl memory.
You can use AWSCore.jl. This package just wraps the AWS REST API which is the same thing that boto does. You therefore shouldn’t notice much of a difference between using the Julia AWS packages and boto.
Because AWS has a REST API for absolutely everything, it’s actually one of the easiest things to use with Julia. For example, I’m probably going to start using some AWS Athena tables soon, which for me is very simple and introduces no dependencies, in contrast to using just about any other kind of database. This is nice because the database situation in Julia is still not very good, one would likely have to resort to using SQLAlchemy through PyCall (since we “lost” JDBC until at least Julia 2.0).
since we “lost” JDBC until at least Julia 2.0
Are you sure? If you’re thinking of https://github.com/JuliaLang/julia/issues/31104 that looks more like something that could be addressed in 1.3 if someone has the time.
I exaggerated a bit. My understanding is that there will be a compile flag for Julia that will make it possible to use JavaCall, but that already drastically reduces the user base. It’s also pretty inconvenient in my case as it complicates deployment, so I probably will avoid JavaCall until I can use it with the pre-compiled binaries.
Please correct me if I’m wrong.
No, Jeff explained the plan for adding a new calling convention here: JVM fails to load in 1.1 (JavaCall.jl) · Issue #31104 · JuliaLang/julia · GitHub
If implemented this looks like it will basically solve the problem with relatively minor caveats and without the need for a command line option.
I’m confused. To me it looks like he’s saying that there will be an ALWAYS_COPY_STACKS build option in 1.2, so if you need to use JavaCall you can build Julia with that option, but that a permanent fix will require a new feature in the ccall interface. I don’t see anything here saying that wouldn’t happen until 2.0, but that was my impression, perhaps that’s wrong.
It sounds to me like that new ccall calling convention is what you need, which presumably could be added by 1.3 since it wouldn’t be breaking. JavaCall would just need to be updated to use this calling convention, which should be as simple as adding a javacall argument to all ccalls into the JVM. Hopefully someone will correct me if this sounds wrong.
1 Like
Exactly, it’s just extending ccall with support for an alternative calling convention. I expect the only thing holding this up is the limited time of the few people who have the right expertise.
1 Like