# Wrapping a Complicated, Multi-Threaded C/C++ Library

**URL:** <https://discourse.julialang.org/t/wrapping-a-complicated-multi-threaded-c-c-library/65401>\
**Category:** Internals & Design\
**Tags:** c, cxxwrap\
**Created:** [July 27, 2021, 9:31pm UTC](https://discourse.julialang.org/t/wrapping-a-complicated-multi-threaded-c-c-library/65401 "2021-07-27T21:31:50Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![colinxs](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/colinxs/32/9071_2.png) [@colinxs](https://discourse.julialang.org/u/colinxs)\
**Post date:** [July 27, 2021, 9:31pm UTC](https://discourse.julialang.org/t/wrapping-a-complicated-multi-threaded-c-c-library/65401/1 "2021-07-27T21:31:51Z")

</div>

I’m attempting to wrap a somewhat complicated C++ library using CxxWrap and am running into issues when it comes to blocking and multi-threaded C++ code. Here’s an abbreviated example of the problem:

```julia
# Functions/objects prefixed with cpp_/Cpp are 
# trivial wrappers around their C++ equivalents.
function loop(object)
    while cpp_is_ready(object)
        cpp_pthread_cond_wait(object) # Calls pthread_cond_wait in C++
        msg = cpp_get_message(object)
        # Do stuff with msg
    end
end

function my_callback(msg)
    # Do stuff with msg
end

function main()
    object = CppObject(@cfunction(my_callback, Nothing, (Cstring, )))
    start(object) # Starts a pthread in C++ 
    Threads.@spawn reader(object)
    request(object, "foobar")
end

```

At a high level, this is a simple reader/writer design. `start(object)` calls a C++ function that initializes a writer thread (using `pthread`) that writes a message to an internal queue for each call to `request(object, msg_request)`. `reader(object)` waits for a message via `cpp_pthread_cond_wait(object)` which internally calls `pthread_cond_wait` in C++ land. `process_message` then grabs that message from the queue and calls the callback `my_function`.

All together, we have three threads:  
Thread 1: `pthread` created in C++ from call to `start(object)`  
Thread 2: Julia thread created from `@spawn`  
Thread 3: Main Julia thread that makes the call to `request(object)`.

and one lock, the `pthread_mutex` created in C++ land.

Now this all works fine, until an exception occurs, either from an error or something like `Ctrl-C`. When that occurs, the process hangs and the terminal becomes unresponsive to input. Not even `Ctrl-C` (SIGINT) or `Ctrl-\` (SIGQUIT) works. I have to close the terminal. Note that issue only occurs when mixing Julia and C++, if I recreate the above code block in pure C++ then `Ctrl-C` works as expected.

Reading through the Julia manual, I found this discussion next to the docs for `@threadcall`:

> If a C library performs a blocking operation, that prevents the Julia scheduler from executing any other tasks until the call returns.

Which leads me to suspect that the issue is unrelated to the multi-threading and solely to do with the call to `cpp_pthread_cond_wait(object)`. Given that the pure C++ version of the above code works find, I suspect that `SIGINT` doesn’t actually get issued so the `reader` that’s waiting on `cpp_pthread_cond_wait` never wakes up. I though about using `@threadcall`, but it is has this caveat:

> It is very important that the called function does not call back into Julia, as it will segfault.

Which disqualifies my since the library requires that I pass it a callback. With this in mind, I tried the following modification:

```julia
function main(object)
    object = CppObject(@cfunction(my_callback, Nothing, (Cstring, )))
    try
        start(object) # Starts a pthread in C++ 
        Threads.@spawn reader(object)
        request(object, "foobar")
    catch e
        # Issue a signal that wakes up the waiting thread in `cpp_pthread_cond_wait`
        issue_signal(object)
    end
end

```

which surprisingly fixes the issue! But it seems like such an ugly hack. **So here’s my question:**

1. Is my deduction that the root cause of the issue is the call to `cpp_pthread_cond_wait` correct/probable?
2. What’s a better solution to the above problem? Can I make sure that `SIGINT` gets signaled somehow? Would that even solve the issue? What about calling `jl_yield` in `cpp_pthread_cond_wait` from C++?

Thank you for any and all help on an incredibly frustrating problem!