# Callback thread safety in 0.6?

**URL:** <https://discourse.julialang.org/t/callback-thread-safety-in-0-6/1291>\
**Category:** General Usage\
**Created:** [January 4, 2017, 9:19pm UTC](https://discourse.julialang.org/t/callback-thread-safety-in-0-6/1291 "2017-01-04T21:19:54Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![ssfrr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ssfrr/32/3736_2.png) [@ssfrr](https://discourse.julialang.org/u/ssfrr)\
**Post date:** [January 4, 2017, 9:19pm UTC](https://discourse.julialang.org/t/callback-thread-safety-in-0-6/1291/1 "2017-01-04T21:19:54Z")

</div>

Does anything in the recent threading development affect [this section](http://docs.julialang.org/en/stable/manual/calling-c-and-fortran-code/?highlight=asynccondition#thread-safety) of the manual? What I’m most interested in is whether we can pass more complex Julia functions to C libraries that might call them from a different context.

I do a bit of this to handle audio callbacks and unfortunately need to do a little more handling from within the callback (mostly copying buffers that are only accessible within the callback context), so I always have to be really careful there’s zero allocation and get a lot of segfaults during development (or if I turn inlining off). It works pretty reliably now (though there’s an issue on Windows), but the code is pretty nasty and hard to maintain, and feels a bit borderline.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [January 5, 2017, 5:53am UTC](https://discourse.julialang.org/t/callback-thread-safety-in-0-6/1291/2 "2017-01-05T05:53:19Z")

</div>

No. \<20 characters\>.

---

<div class="post-metadata">

**Author:** ![twadleigh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/twadleigh/32/22778_2.png) [@twadleigh](https://discourse.julialang.org/u/twadleigh)\
**Post date:** [April 19, 2017, 11:32pm UTC](https://discourse.julialang.org/t/callback-thread-safety-in-0-6/1291/3 "2017-04-19T23:32:19Z")

</div>

@ssfrr, could you point to an example of where you do this (do the threadsafe callback thing)? I am trying to decode that section of the manual right now, and could use an actual example from which to work.

---

<div class="post-metadata">

**Author:** ![ssfrr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ssfrr/32/3736_2.png) [@ssfrr](https://discourse.julialang.org/u/ssfrr)\
**Post date:** [April 20, 2017, 4:02am UTC](https://discourse.julialang.org/t/callback-thread-safety-in-0-6/1291/4 "2017-04-20T04:02:51Z")

</div>

My packages probably aren’t the best example, as I’m doing some hacky things to push the data into ringbuffers within the callbacks. I’m actually going to be re-architecting that part pretty soon so that it’s less brittle.

Here’s the general idea though. Say you have a C library `libjepsen` that lets you register a callback:

```nohighlight
void call_me_maybe(int (*callback)(int, void*), void *context)

```

So it’s expecting a callback of the form:

```nohighlight
int handle_callback(int mynumber, void *context)

```

We’ll assume here that it’s handing you back some kind of data in the `mynumber` parameter it thinks you might be interested in, as well as whatever `context` pointer you passed in when you registered the callback.

In Julia you’d use this library with something like this:

```julia
cond = AsyncCondition()

# this function CANNOT do anything that interacts with the GC
# note we're assuming that our condition handle is being passed in as context
function callback(number, context)
    ccall(:uv_async_send, Cint, (Ptr{Void}, ), context)
    return 0
end

c_callback = cfunction(callback, Cint, (Cint, Ptr{Void})

# register our callback with the condition handle as the
# context pointer so we can use it from within the callback
ccall((:call_me_maybe, libjepsen),
      Void,
      (Ptr{Void}, Ptr{Void}),
      c_callback,
      cond.handle)

while true
    wait(cond)
    println("this is crazy")
end

```

Note that if the callback gets called multiple times before the Julia-side scheduler gets around to waking up `cond`, the `while` loop will only print once. Also note that this doesn’t provide a good mechanism to pass data between thread contexts. I wrote [RingBuffers.jl](https://github.com/JuliaAudio/RingBuffers.jl) which lets you read and write from the other thread context, but it makes some assumptions about what’s “safe” to do from the other thread and may be a little dicey. It also needs a documentation update - LMK if you want to use it and I’ll help out.

---

<div class="post-metadata">

**Author:** ![twadleigh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/twadleigh/32/22778_2.png) [@twadleigh](https://discourse.julialang.org/u/twadleigh)\
**Post date:** [April 20, 2017, 9:22pm UTC](https://discourse.julialang.org/t/callback-thread-safety-in-0-6/1291/5 "2017-04-20T21:22:56Z")

</div>

Thanks for the self-contained example.

One of the things I didn’t know how to deal with was being able to pass data, rather than just executing a thunk. Unwinding julia library code and going though libuv documentation, I figured out that the `uv_async_t` instance that `AsyncCondition.handle` points to has a `data` field that can be set on the C side before `uv_async_send` is invoked. On the Julia side, `uv_handle_data` gives you the pointer that was set. It would take some further synchronization machinery to construct an actual blocking function call with a return value.

The use case I have in mind isn’t really the callback problem, but calling into the Julia runtime from an embedding application. I want to be able to invoke Julia from different threads in the application. I believe the problem and its solution are analogous to this, though.

Thanks again.

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [April 20, 2017, 9:36pm UTC](https://discourse.julialang.org/t/callback-thread-safety-in-0-6/1291/6 "2017-04-20T21:36:18Z")

</div>

I had to deal with a usecase where I had to actually pass data along from the callback [https://github.com/JuliaGPU/OpenCL.jl/blob/52a6e701183a0324a408db0ae7bb9be00d3037e8/src/event.jl#L97-L140](https://github.com/JuliaGPU/OpenCL.jl/blob/52a6e701183a0324a408db0ae7bb9be00d3037e8/src/event.jl#L97-L140)

---

<div class="post-metadata">

**Author:** ![twadleigh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/twadleigh/32/22778_2.png) [@twadleigh](https://discourse.julialang.org/u/twadleigh)\
**Post date:** [April 21, 2017, 12:02am UTC](https://discourse.julialang.org/t/callback-thread-safety-in-0-6/1291/7 "2017-04-21T00:02:43Z")

</div>

@vchuravy, just to confirm I’m understanding the code you posted:

- the callback is good for one invocation only;
- it doesn’t return a value or block until the “real” code is done, yes?

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [April 21, 2017, 12:12am UTC](https://discourse.julialang.org/t/callback-thread-safety-in-0-6/1291/8 "2017-04-21T00:12:28Z")

</div>

Yes this version is only good for one invocation only (which is necessary for this particular usecase). I presume that you mean blocking from the C++ side? I tried to implement that once for a different project using libuv mutexes and barriers. If you have control over the C++ side you should be able to create a libuv barrier and pass it to the callback and then wait on it being unlocked from your Julia code. But you won’t be able to reuse your `AsyncCallback` since those can get lost (if another one fires, before your code get to the previous one)