We are working on SaaS based platform implemented using Flask and Celery.
In the backend, we have some part of the implementation written in julia.
We integrated julia using a python library called julia==0.6.0
In the flask we are able to run the julia code.
However we are not able to run the same code inside the celery task. The error that we get is
celery_worker_1 | signal (4): Illegal instruction
celery_worker_1 | in expression starting at none:2
celery_worker_1 | unknown function (ip: 0xffff727d6cec)
celery_worker_1 | _jl_invoke at /cache/build/default-awsarm-5/julialang/julia-release-1-dot-8/src/gf.c:2365 [inlined]
celery_worker_1 | ijl_apply_generic at /cache/build/default-awsarm-5/julialang/julia-release-1-dot-8/src/gf.c:2547
celery_worker_1 | Allocations: 3462340 (Pool: 3459082; Big: 3258); GC: 4
celery_worker_1 | [2022-12-18
We are using julia v1.8.3 (even we have tried v1.9.0-alpha1 but we are getting the same error).
We integrated julia using a python library called julia==0.6.0
It seems you’re using PyJulia package, but note its docs, under “Limitations” (your error is likely a race condition because of the given limitation of the software you use):
No threading support
PyJulia cannot be used in different threads since libjulia is not thread safe. However, you can use multiple threads within Julia. For example, start IPython by JULIA_NUM_THREADS=4 ipython and then run:
In [1]: %load_ext julia.magic Initializing Julia runtime. This may take some time… In [2]: %%julia …: a = zeros(10) …: Threads.@threads for i = 1:10 …: a[i] = Threads.threadid() …: end …: a Out[3]: array([1., 1., 1., 2., 2., 2., 3., 3., 4., 4.])
PyJulia does not release GIL
PyJulia does not release the Global Interpreter Lock (GIL) while calling Julia functions since PyCall expects the GIL to be acquired always. It means that Python code and Julia code cannot run in parallel.
I recognize the error you have, as something coming from Julia without the “celery_worker_1 |” prefix, which I believe the Python side added somehow. I suppose that’s celery worker 1 (of many), i.e. a thread?
You can use all Python libraries from Julia (i.e. Julia is your main language) through PythonCall.jl (or older PyCall.jl). With it or PyJulia, you can also use all Julia libraries from Python. In both cases some limitations may apply:
When you e.g. call Julia from Python, the library calling may also be a framework. Frameworks call you (“the Hollywood principle” of frameworks), so they must run in the “top” language, as you are doing with Flask.
You may want to consider some popular Julia (web) framework such as Genie.jl as a replacement for Python’s Flask (and replace all Python framework stuff), or e.g. this alternative:
Because serving web pages means a lot of requests come in and you want to serve them as fast as possible, meaning in parallel (“asynchronous”), often done with threading, something those interop libraries may not support. Unless possible if you jump through some hoops. Something you can avoid if using a full Julia (or full Python) solution.
Asynchronous Julia code will not normally run while Python is executing, unless it is in a separate thread.
This can be fixed by calling jl.yield() periodically from Python code, allowing the Julia scheduler to run.
When working at the Python REPL, you may call juliacall.interactive() which will allow Julia async code to run while the prompt is showing. This will allow interactive plots such as Makie to work.
If you can disable threading somehow (it would be slower), just to test, and you get rid of your error, then threading is the most likely, expected, cause of your problem. You would be safe to run your code that way, if you can tolerate losing out on threading, at least until you migrate to a better solution.
I had heard of Flask, but not Celery. The latter seems to be to handle long-running task from Flask, i.e. not in the same thread, likely spawns its own. You may need to disable Celery or its threading (which seems its main point). Possibly if Celery is just used on the Python side, not for calling Julia then maybe you don’t need to interfere with it (despite the error message pointing to it). But I very much doubt you get away with keeping Celery unchanged, and it might be code dependant, possibly ok for some of your code, but not other Celery using code.
To answer the initial question of @shwetank_verma, I recommend configuring backend in DanceJL, looking at limited endpoints from your code. You can easily add a simple HTTP Bearer check if necessary.