Julia function from non-Julia thread and pthread_join

General Usage
, ,
1

Since 46609 has been merged, it has become possible to have non-Julia threads call Julia functions. Accordingly, I have set up the following MWE of a C++ library callback_example compiled with g++ -shared -o callback_example.so -fPIC -pthread callback_example.cpp:

#include <pthread.h>
#include <iostream>

// Define the callback function type
typedef void (*CallbackFunction)();

// Define the thread function that executes the callback
extern "C" void* threadFunc(void* arg)
{
    CallbackFunction callback = reinterpret_cast<CallbackFunction>(arg);
    std::cout << "Thread started.\n";
    callback();
    std::cout << "Thread ended.\n";
    return nullptr;
}

// Define the function that takes the callback and executes it in another thread
extern "C" void executeCallbackInThread(CallbackFunction callback)
{
    // Create a new thread and execute the callback in it
    pthread_t thread;
    pthread_create(&thread, nullptr, &threadFunc, reinterpret_cast<void*>(callback));
}

Setting up a Julia interface according to

lib = joinpath(pwd(), "callback_example")

function callback()
    println("Sample callback executed.")
end

function threaded_callback()
    callback = @cfunction(callback, Cvoid, ())
    ccall((:executeCallbackInThread, lib), Cvoid, (Ptr{Cvoid},), callback)
end

yields the desired output

julia> threaded_callback(); sleep(0.1)
Thread started.
Sample callback executed.
Thread ended.

However, if I add another C++ library function that waits for the pthread to finish, i.e.,

extern "C" void executeCallbackInThreadJoin(CallbackFunction callback)
{
    // Create a new thread and execute the callback in it
    pthread_t thread;
    pthread_create(&thread, nullptr, &threadFunc, reinterpret_cast<void*>(callback));
    // Now wait for the thread to finish -> this breaks julia execution
    pthread_join(thread, nullptr);
}

recompile the library and add the corresponding julia interface

function threaded_callback_join()
    callback = @cfunction(callback, Cvoid, ())
    ccall((:executeCallbackInThreadJoin, lib), Cvoid, (Ptr{Cvoid},), callback)
end

the execution gets stuck and won’t terminate without interrupting.

julia> threaded_callback_join(); sleep(0.1)
Thread started.
Sample callback executed.

However, if I replace the callback function by a non-printing one, e.g., callback() = nothing, the execution does not get stuck. Is there any way around this behavior? For the sake of convenience, I have attached the source code of the C-library (I have added the jl extension to allow for the upload) and a julia script comprising all the test cases I have mentioned.

Thanks a lot!
callback_example.cpp.jl (1.1 KB)
callback_example_interface.jl (967 Bytes)