# Not able to run Julia through Flask-Celery
**URL:** https://discourse.julialang.org/t/not-able-to-run-julia-through-flask-celery/91849
**Category:** General Usage
**Tags:** web
**Created:** [December 19, 2022, 12:40pm UTC](https://discourse.julialang.org/t/not-able-to-run-julia-through-flask-celery/91849 "2022-12-19T12:40:36Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@shwetank\_verma](https://discourse.julialang.org/u/shwetank_verma)
#### Post date: [December 19, 2022, 12:40pm UTC](https://discourse.julialang.org/t/not-able-to-run-julia-through-flask-celery/91849/1 "2022-12-19T12:40:36Z")
We are working on SaaS based platform implemented using Flask and Celery.
In the backend, we have some part of the implementation written in julia.
We integrated julia using a python library called julia==0.6.0
In the flask we are able to run the julia code.
However we are not able to run the same code inside the celery task. The error that we get is
celery\_worker\_1 | signal (4): Illegal instruction
celery\_worker\_1 | in expression starting at none:2
celery\_worker\_1 | unknown function (ip: 0xffff727d6cec)
celery\_worker\_1 | \_jl\_invoke at /cache/build/default-awsarm-5/julialang/julia-release-1-dot-8/src/gf.c:2365 [inlined]
celery\_worker\_1 | ijl\_apply\_generic at /cache/build/default-awsarm-5/julialang/julia-release-1-dot-8/src/gf.c:2547
celery\_worker\_1 | Allocations: 3462340 (Pool: 3459082; Big: 3258); GC: 4
celery\_worker\_1 | [2022-12-18
We are using julia v1.8.3 (even we have tried v1.9.0-alpha1 but we are getting the same error).
Here’s the repo for a reproducible example: [GitHub - p-oorja/p-oorja: Config files for my GitHub profile.](https://github.com/p-oorja/p-oorja)
---
### Author:  [@josuagrw](https://discourse.julialang.org/u/josuagrw)
#### Post date: [December 20, 2022, 11:24am UTC](https://discourse.julialang.org/t/not-able-to-run-julia-through-flask-celery/91849/2 "2022-12-20T11:24:15Z")
Welcome to the Julia discourse!
> [@Please read: make it easier to help you](https://discourse.julialang.org/t/please-read-make-it-easier-to-help-you/14757):
>
> Welcome to the Julia Discourse! We are enthusiastic about helping Julia programmers, both beginner and experienced. This public service announcement (PSA) outlines best practices when asking for help. Following these points makes it easier for us to help you and more likely you’ll get a prompt, useful answer. Keywords are highlighted to make it easier to refer to specific points. Choose a descriptive title that captures the key part of your question, eg “plots with multiple axes” instead of …
---
### Author:  [@Prashant](https://discourse.julialang.org/u/Prashant)
#### Post date: [January 5, 2023, 8:50am UTC](https://discourse.julialang.org/t/not-able-to-run-julia-through-flask-celery/91849/3 "2023-01-05T08:50:18Z")
Has anybody got solution to this…
---
### Author:  [@josuagrw](https://discourse.julialang.org/u/josuagrw)
#### Post date: [January 5, 2023, 12:32pm UTC](https://discourse.julialang.org/t/not-able-to-run-julia-through-flask-celery/91849/4 "2023-01-05T12:32:07Z")
I suspect the users of a Celery / Python forum might be more equipped to answer your question.
---
### Author:  [@Palli](https://discourse.julialang.org/u/Palli)
#### Post date: [January 22, 2023, 4:54pm UTC](https://discourse.julialang.org/t/not-able-to-run-julia-through-flask-celery/91849/5 "2023-01-22T16:54:31Z")
> We integrated julia using a python library called julia==0.6.0
It seems you’re using PyJulia package, but note its docs, under “Limitations” (your error is likely a race condition because of the given limitation of the software you use):
> ## No threading support
>
> PyJulia cannot be used in different threads since libjulia is not thread safe. However, you can [use multiple threads within Julia](https://docs.julialang.org/en/v1/manual/parallel-computing/#Multi-Threading-(Experimental)-1). For example, start IPython by `JULIA_NUM_THREADS=4 ipython` and then run:
>
> In [1]: %load\_ext julia.magic Initializing Julia runtime. This may take some time… In [2]: %%julia …: a = zeros(10) …: Threads.@threads for i = 1:10 …: a[i] = Threads.threadid() …: end …: a Out[3]: array([1., 1., 1., 2., 2., 2., 3., 3., 4., 4.])
>
> ## PyJulia does not release GIL
>
> PyJulia does not release the Global Interpreter Lock (GIL) while calling Julia functions since PyCall expects the GIL to be acquired always. It means that Python code and Julia code cannot run in parallel.
I recognize the error you have, as something coming from Julia without the “celery\_worker\_1 |” prefix, which I believe the Python side added somehow. I suppose that’s celery worker 1 (of many), i.e. a thread?
You can use all Python libraries from Julia (i.e. Julia is your main language) through PythonCall.jl (or older PyCall.jl). With it or PyJulia, you can also use all Julia libraries from Python. In both cases some limitations may apply:
When you e.g. call Julia from Python, the library calling may also be a framework. Frameworks call you (“the Hollywood principle” of frameworks), so they must run in the “top” language, as you are doing with Flask.
You may want to consider some popular Julia (web) framework such as Genie.jl as a replacement for Python’s Flask (and replace all Python framework stuff), or e.g. this alternative:
> [@Dance.jl - Python Flask equivalent web framework](https://discourse.julialang.org/t/dance-jl-python-flask-equivalent-web-framework/38560):
>
> Hello Julia community, Am pleased to announce a new web framework focused on facilitating process of outputting Julia computed data to the web very easily. Feel that reason Python is still more mainstream than Julia is simply due to complete ecosystem of tools. More precisely many data scientists can easily use Flask to share their data with outside world. Not to that mention staying with 1 backend language is simply easier for many large teams. Which is why I have developed Dance.jl along wit…
Because serving web pages means a lot of requests come in and you want to serve them as fast as possible, meaning in parallel (“asynchronous”), often done with threading, something those interop libraries may not support. Unless possible if you jump through some hoops. Something you can avoid if using a full Julia (or full Python) solution.
> [@Dance.jl - Python Flask equivalent web framework](https://discourse.julialang.org/t/dance-jl-python-flask-equivalent-web-framework/38560):
>
> Hello Julia community, Am pleased to announce a new web framework focused on facilitating process of outputting Julia computed data to the web very easily. Feel that reason Python is still more mainstream than Julia is simply due to complete ecosystem of tools. More precisely many data scientists can easily use Flask to share their data with outside world. Not to that mention staying with 1 backend language is simply easier for many large teams. Which is why I have developed Dance.jl along wit…
> **[Compatibility Tools · PythonCall & JuliaCall](https://cjdoris.github.io/PythonCall.jl/stable/compat/)**
>
> Documentation for PythonCall & JuliaCall.
> ## [Asynchronous Julia code (including Makie)](https://cjdoris.github.io/PythonCall.jl/stable/compat/#Asynchronous-Julia-code-(including-Makie))
>
> Asynchronous Julia code will not normally run while Python is executing, unless it is in a separate thread.
>
> This can be fixed by calling `jl.yield()` periodically from Python code, allowing the Julia scheduler to run.
>
> When working at the Python REPL, you may call `juliacall.interactive()` which will allow Julia async code to run while the prompt is showing. This will allow interactive plots such as Makie to work.
If you can disable threading somehow (it would be slower), just to test, and you get rid of your error, then threading is the most likely, expected, cause of your problem. You would be safe to run your code that way, if you can tolerate losing out on threading, at least until you migrate to a better solution.
>
I had heard of Flask, but not Celery. The latter seems to be to handle long-running task from Flask, i.e. not in the same thread, likely spawns its own. You may need to disable Celery or its threading (which seems its main point). Possibly if Celery is just used on the Python side, not for calling Julia then maybe you don’t need to interfere with it (despite the error message pointing to it). But I very much doubt you get away with keeping Celery unchanged, and it might be code dependant, possibly ok for some of your code, but not other Celery using code.
---
### Author:  [@yoh-meyers](https://discourse.julialang.org/u/yoh-meyers)
#### Post date: [January 24, 2023, 7:43am UTC](https://discourse.julialang.org/t/not-able-to-run-julia-through-flask-celery/91849/6 "2023-01-24T07:43:16Z")
Thank you for mentioning my package @Palli .
To answer the initial question of @shwetank_verma, I recommend configuring backend in DanceJL, looking at limited endpoints from your code. You can easily add a simple HTTP Bearer check if necessary.
Also there is no equivalent of a Celery library in Julia. So why not use something such as AWS MQ, with [GitHub - rweilbacher/MQTT.jl: An asynchronous MQTT client library for julia](https://github.com/rweilbacher/MQTT.jl). That would be fastest way moving forward, in my opinion.