Sometime ago I asked about calling Julia in multiple threads. It was said that this is on roadmap but with low priority, IIRC.
Something that would help me a lot would be the possibility to have multiple Julia instances in the same C++ software. Something like Lua. My use case is:
One thread runs a simulator that communicates with the main software by TCP maybe.
A separate Julia thread gets the simulator data and performs some analysis on demand.
Since it is not possible today, what I am doing is to run the simulator on terminal, outside the C++ software.
My question is: is such feature on the roadmap for Julia?
I understand that such use cases exist, but I’m tempted to respond with “can I run multiple instances of a C++ library within julia?”
I think the way forward here would be to try to leverage julia’s multithreading. If the threads are started by C++, we could add an API to notify julia about the threads so it can execute on them. Does that sound sufficient?
In fact, that will solve all my problems! My proposal about multiple instances was because of my limited knowledge about the system. I thought that, if Julia cannot be called from multiple threads, then maybe it would be easier to have multiple instances, one in each thread.
Funny that in the past days I’ve been looking into Julia threading C code and trying to create a way to “grow” jl_n_threads after Julia had already been initialized so that an external thread could register and initialize its Julia TLS. I’m now trying to understand what is done for each native Julia thread during jl_init() that must be done for the new threads too.
EDIT: Let me add that this is no serious effort, it’s probably way over my competences or available time . Also, it seems it would make more sense to distinguish Julia managed threads and external ones anyway.
The difficulty started out as “extreme - not possible in this version”, but I expect it’s been decreasing a lot recently as the runtime becomes more threadsafe. I can’t say I know how hard it is right now; at a guess, “within grasp but difficult” might cover it Allocating new julia-specific TLS and making the runtime aware of that seems ok, but I don’t know how many other assumptions are made about thread ownership. Grepping for jl_n_threads seems somewhat informative.
Funny that in the past days I’ve been looking into Julia threading C code and trying to create a way to “grow” 
. Also, it seems it would make more sense to distinguish Julia managed threads and external ones anyway.
Allocating new julia-specific TLS and making the runtime aware of that seems ok, but I don’t know how many other assumptions are made about thread ownership. Grepping for