Julia wrapper for hiredis

General Usage
1

Hi everyone,

I am currently trying to write a Julia wrapper for the C library hiredis, so that I can write data to and read data from a Redis Cache with Julia.

There are already two Julia libraries for this, namely Redis.jl and Jedis.jl, however, one of them does not have TLS support and the other does not seem to work properly. Also, I would like to learn more about calling C libraries from Julia.

This is what I have so far:

I have a Redis Cache instance running in the background in a docker container.

docker run -p 6379:6379 --name some-redis -d redis

Using a Python script, I verified that the instance is running correctly.

import redis

r = redis.Redis(host = "localhost", port = 6379)
r.ping()

Now, I am trying to do the same thing in Julia:

using hiredis_jll

context = @ccall libhiredis.redisConnect("localhost"::Cstring, 6379::Cint)::RedisContext
reply = @ccall libhiredis.redisCommand(context::RedisContext, "PING"::Cstring)::RedisReply

(This code is inspired by the examples from the hiredis repository: hiredis/examples at master · redis/hiredis · GitHub)

Of course, I also need to define the RedisReply and the RedisContext type in order for this to work:

using StaticArrays

struct RedisReply
    type::Cint
    integer::Int64
    dval::Int64
    len::Int64
    str::Int64
    vtype::Cint
    elements::Int64
    element::Int64
end

struct TCP
    host::Int64
    source_addr::Int64
    port::Cint
end

struct UnixSock
    path::Int64
end

struct RedisContextFuncs
    close::Int64
    free_privctx::Int64
    async_read::Int64
    async_write::Int64
    read::Int64
    write::Int64
end

struct RedisContext
    funcs::RedisContextFuncs
    err::Cint
    errstr::SVector{128,UInt8}
    fd::Cint
    flags::Cint
    obuf::Int64
    reader::Int64
    connection_type::Int64
    connect_timeout::Int64
    command_timout::Int64
    tcp::TCP
    unix_sock::UnixSock
    saddr::Int64
    addrlen::Int64
    privdata::Int64
    free_privdata::Int64
    privctx::Int64
    push_cb::Int64
end

However, when I try to run the above code I get the following error (more precisely, when libhiredis.redisCommand is invoked):

signal (11): Segmentation fault
in expression starting at /mnt/data/hiredis/hiredis.jl:66
sdslen at /workspace/srcdir/hiredis/sds.h:94 [inlined]
sdscatlen at /workspace/srcdir/hiredis/sds.c:382
jl_typeinf_world at /opt/julia-1.8.5/bin/../lib/julia/libjulia-internal.so.1 (unknown line)
unknown function (ip: 0x20fd287)
Allocations: 2906 (Pool: 2895; Big: 11); GC: 0
fish: Job 1, 'julia --project hiredis.jl' terminated by signal SIGSEGV (Address boundary error)

Also, inspecting the RedisContext that is created after libhiredis.redisConnect is invoked reveals that there might be an issue already with that call since context.err is apparently not equal to 0.

julia> context = @ccall libhiredis.redisConnect("localhost"::Cstring, 6379::Cint)::RedisContext
RedisContext(RedisContextFuncs(32459, 140048809884736, 140048680454784, 140045947296448, 140046120224464, 140048820755589), 121603608, UInt8[0x5f, 0x7f, 0x00, 0x00, 0xcb, 0x7e, 0x00, 0x00, 0x00, 0x00  …  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, 0x44, 0xb9, 0x0a], 32607, -1802244432, 140046120224560, 140048807005686, 0, 140048807114847, 140048680454784, TCP(140048486337312, 140048486336720, -1468035016), UnixSock(140046178534592), 0, 0, 0, 140046120224656, 140048807008866, 140046120225216)

julia> context.err
121603608

My guess is that there is some problem with mapping the returned C object to the Julia types. These types are relatively complex, and notice that I am not using the same types in the Julia type definitions (see here how redisContext and redisReply are defined in hiredis: hiredis/hiredis.h at master · redis/hiredis · GitHub). But is that really necessary? I thought that it might be sufficient to have the same amount of memory per field in the corresponding Julia type.

Given that it would require a lot of work to make sure that the C types and Julia types match exactly, I first wanted to make sure whether there might be another issue with what I have so far.

1 Like
2

This likely needs to be a NTuple{128, Cchar}.

This is not an Int64 in the linked hiredis.h, but a char*, i.e. a Ptr{Cchar} on the julia side.

There are likely other issues like that with your structs, see here

https://docs.julialang.org/en/v1/manual/calling-c-and-fortran-code/#man-bits-types

3

Thanks, I tried to define the Julia types as closely as possible to the C types based on the link that you provided.

What was ultimately missing to make the code work, however, was to treat the return values of the C calls as pointers:

using hiredis_jll

context_pointer = @ccall libhiredis.redisConnect("localhost"::Cstring, 6379::Cint)::Ptr{RedisContext}
reply_pointer = @ccall libhiredis.redisCommand(context_pointer::Ptr{RedisContext}, "PING"::Cstring)::Ptr{RedisReply}

reply = unsafe_load(reply_pointer)
unsafe_string(reply.str) # "PONG"

Note that the code snippet above even works when you replace RedisContext by Cvoid, for example. Thus, having the right mapping between Julia and C types does not seem to be that important.