Unknown error when using ccall to work with Apache Arrow's GLib interface

Hey all,

Apache Arrow

  • is an in memory columnar format
  • has some very speedy IO implementations in c++
  • has a C interface using GLib

Since it’s an in-memory format, I’d like to perform IO in cpp, and then convert the resulting c++ Arrow Dataset into an Arrow.jl dataset.

When I run this code, jupyter crashes without too many details

using Pkg
using Gtk.GLib
path = "file.parquet"
GError() do error_check
    x = ccall(
        (:gparquet_arrow_file_reader_new_path, "/usr/local/Cellar/apache-arrow-glib/2.0.0/lib/libparquet-glib.200"), 
        (AbstractString, Ptr{GError}), 
        path, error_check

I suspect it has something to do with the Types AbstractString and Ptr{GError} but I’m not sure how to debug. Does anyone have ideas?

It would also be nice if I could code-gen the GLib bindings into Julia. I’ve never done this sort of thing before so I would really appreciate some pointers

On second thought - this fails too:

GError() do error_check

with this error message

MethodError: no method matching !(::Nothing)
Closest candidates are:
  !(!Matched::Missing) at missing.jl:100
  !(!Matched::Bool) at bool.jl:33
  !(!Matched::Function) at operators.jl:896

 [1] GError(::var"#3#4")....

I don’t really know if there’s a guide to use GLib in Julia but if there is one, I would be happy to read through it

Looking into Gtk.jl code it seems it’s usually used with a pointer to pointer, which is also what you have in the doc of gparquet_arrow_file_reader_new_path.

As for the string you have to declare it as Ptr{UInt8} and convert your string using Gtk.bytestring (although I think it does nothing in your case).

1 Like

Thank you very much!!
I’m still seeing an error from GError() do error_check ... but I don’t think it’s easy to get around that