# Embedding Julia while maintaining task scheduler

**URL:** https://discourse.julialang.org/t/embedding-julia-while-maintaining-task-scheduler/24851
**Category:** Internals & Design
**Tags:** question, embedding
**Created:** [June 2, 2019, 11:34am UTC](https://discourse.julialang.org/t/embedding-julia-while-maintaining-task-scheduler/24851 "2019-06-02T11:34:38Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![cdsousa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cdsousa/32/215553_2.png) [@cdsousa](https://discourse.julialang.org/u/cdsousa)
#### Post date: [June 2, 2019, 11:34am UTC](https://discourse.julialang.org/t/embedding-julia-while-maintaining-task-scheduler/24851/1 "2019-06-02T11:34:38Z")

</div>

Hi, I’ve been trying to be able to embed Julia in a C++ application and access it from multiple (C++) threads (Why? [Some context here](http://julia-programming-language.2336112.n4.nabble.com/A-quot-two-languages-quot-problem-when-embedding-Julia-td48875.html)).

After start learning how Julia “managed” threads work, I start trying to add methods to register an external thread as a Julia “unmanaged” one. However, at some point, I realized that the easiest way would be to have the main thread in Julia (a dedicated C++ thread where Julia has been initialized) accepting tasks and scheduling to other “managed” threads as needed. The tasks could be queued and waited for from any C++ thread since direct calls into the Julia runtime would be transferred to and run in the main thread.

Now the problem is that I have not being able to have the main Julia thread waiting for C++ tasks, using a C++ condition variable for instance, and at the same time having the Julia running its scheduler in that very same thread. I can either be waiting on the C++ condition or on the Julia scheduler, but not both.

[I’ve been doing several experiments](https://github.com/cdsousa/embedding_julia/blob/master/julia_cpp_ts.cpp), and looked into using Libuv condition variables and streams, but without success.

I think I need a way to have Julia in its internal loop but while having a hook that can be triggered from the C++ side.  
Does anyone have any hint or advice on I can solve this problem? Thanks.

---

<div class="post-metadata">

### Author: ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)
#### Post date: [June 2, 2019, 1:53pm UTC](https://discourse.julialang.org/t/embedding-julia-while-maintaining-task-scheduler/24851/2 "2019-06-02T13:53:59Z")

</div>

I don’t know enough about Julia internals but I know a bit about threading. the Julia tasks are not threads so the main Julia loop can’t schedule tasks and block on a thread sync primitive. I’d suggest to create a C++ thread that waits on a thread sync primitive and then all a Julia task has to do is fire it… which shouldn’t ever block.

---

<div class="post-metadata">

### Author: ![cdsousa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cdsousa/32/215553_2.png) [@cdsousa](https://discourse.julialang.org/u/cdsousa)
#### Post date: [June 2, 2019, 11:25pm UTC](https://discourse.julialang.org/t/embedding-julia-while-maintaining-task-scheduler/24851/3 "2019-06-02T23:25:36Z")

</div>

> [@dlakelan](#):
>
> Julia tasks are not threads

Thanks. I was not clear, but I am actually aiming at Julia new multithreaded task scheduler (`task.sticky = false`).

What I want, in the end, is to be able to have several C++ parallel threads running and calling into Julia for running several Julia “tasks” in parallel.

I already solved the C++ code waiting for its Julia task to finish, by using condition variables that are notified from Julia with ccalls.

---

<div class="post-metadata">

### Author: ![cdsousa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cdsousa/32/215553_2.png) [@cdsousa](https://discourse.julialang.org/u/cdsousa)
#### Post date: [June 8, 2019, 11:05pm UTC](https://discourse.julialang.org/t/embedding-julia-while-maintaining-task-scheduler/24851/4 "2019-06-08T23:05:08Z")

</div>

Well, I finally solved this. The solution is the use of a Julia’s `AsyncCondition` (instead of a C++ `std::condition_variable`), which seems to have been created to solve problems similar to mine: [Calling C and Fortran Code · The Julia Language](https://docs.julialang.org/en/v1.1/manual/calling-c-and-fortran-code/#Thread-safety-1).
