Not able to run Julia through python on a server in a frontend type application

Hey everyone,

I am not able to find anything on this so I will try to explain as many things as I can.

We are building a cloud-based simulation application whose frontend is Javascript which is calling python script. The simulation code is written in Julia (differentialequation.jl) which is wrapped in python (3.8). These wrappers are being called by the frontend code.

This python3.8 was installed without libpython enabled. Hence I have been using these lines

    from julia.api import Julia
    jl = Julia(compiled_modules=False)
    from julia import Main

in the python code.

When I run things in the terminal everything runs fine. I can run Julia and Julia in python in any directory without any issues.

The issue happens when we try to run things from the frontend GUI. We get this error:

stderr: Traceback (most recent call last):
  File "/home/oorja/Test_project/capacity_fade_julia/run_train.py", line 10, in <module>
    out = estimate_cell_parameters(data)
  File "/home/oorja/Test_project/capacity_fade_julia/integration_function.py", line 70, in estimate_cell_parameters
    return train(train_input)
  File "/home/oorja/Test_project/capacity_fade_julia/train_and_predict.py", line 208, in train
    jl = Julia(compiled_modules=False)
  File "/home/oorja/.local/lib/python3.8/site-packages/julia/core.py", line 468, in __init__
    jlinfo = JuliaInfo.load(runtime)
  File "/home/oorja/.local/lib/python3.8/site-packages/julia/juliainfo.py", line 68, in load
    proc = subprocess.Popen(
  File "/usr/lib/python3.8/subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.8/subprocess.py", line 1702, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'julia'

I have manually checked that the Julia directory is in the folder that is giving the error. The error only comes when running the code through the UI which creates a separate user session. However, we can call Julia in a python virtual environment without any issues.

I tried using a system image. However, it’s not working:

Python 3.8.0 (default, Dec  9 2021, 17:53:27)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from julia import Julia
>>> jl = Julia(sysimage="sys.so")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/oorja/.local/lib/python3.8/site-packages/julia/core.py", line 696, in __init__
    self.__julia = Julia(*args, **kwargs)
  File "/home/oorja/.local/lib/python3.8/site-packages/julia/core.py", line 483, in __init__
    raise UnsupportedPythonError(jlinfo)
julia.core.UnsupportedPythonError: It seems your Julia and PyJulia setup are not supported.

Julia executable:
    julia
Python interpreter and libpython used by PyCall.jl:
    /home/oorja/.julia/conda/3/bin/python
    /home/oorja/.julia/conda/3/lib/libpython3.10.so.1.0
Python interpreter used to import PyJulia and its libpython.
    /usr/bin/python3
    None

Your Python interpreter "/usr/bin/python3"
is statically linked to libpython.  Currently, PyJulia does not fully
support such Python interpreter.

The easiest workaround is to pass `compiled_modules=False` to `Julia`
constructor.  To do so, first *reboot* your Python REPL (if this happened
inside an interactive session) and then evaluate:

    >>> from julia.api import Julia
    >>> jl = Julia(compiled_modules=False)

Another workaround is to run your Python script with `python-jl`
command bundled in PyJulia.  You can simply do:

    $ python-jl PATH/TO/YOUR/SCRIPT.py

See `python-jl --help` for more information.

For more information, see:

    https://pyjulia.readthedocs.io/en/latest/troubleshooting.html

Julia’s version is 1.7.0
Please help.

Two damn unuseful questions from me.

a) Is this the difference between interactive and non-interactive login shells? You do get a slightly different environment with a non-interactive login

b) Have you looked at Pluto notebooks? I guess they may not be relevant to what you are constructing.

We have fixed this issue now. I am leaving this here for anyone who gets this issue.

Essentially what happened was that I had installed Julia 1.8 in the beginning. But I had some issues with it so I uninstalled it, deleted the folder, and removed the environment variables. Then I installed Julia 1.7 and all the required libraries. I also added it to the environment variables. I tested it many times and it worked fine.

But, when the frontend script was calling the Julia code, it was calling 1.8 and not 1.7. I don’t know how this happened but always be sure that your frontend script is calling the correct version.

This error was also not obvious from the stack trace and hence it took us some time to figure it out.