Any working way to call Python from Julia compiled app?

I tried to compile a relocatable app with PackageCompiler.create_app and call Python from it.

  1. With PyCall.jl - there is a problem that compiled binary has an absolute path to python library on dev machine (on my Windows machine it is C:\Users\gvg\.julia\conda\3\python39.dll), so when I’m on another machine with different user - it can’t find the python by this path. See issue here:
  2. With PythonCall.jl - there is another error after compilation, looks like it can’t find Manifes.toml in compiled project files. See issue here:

Any other ways I’ve missed?
With option 1, maybe I can set another python path before compilation, so it is not bound to a specific user?

I’m also finding PythonCall and PackageCompiler not working well together. It doesn’t use the JULIA_PYTHONCALL_EXE environment variable when compiled. For large sysimage with all packages included, it failed all together when starting julia with the compiled image:

fatal: error thrown and no exception handler available.
InitError(mod=:C, error=AssertionError(msg=“CTX.which == :PyCall”))
#34 at ….julia\packages\PythonCall\XgP8G\src\cpython\context.jl:135
with_gil at ….julia\packages\PythonCall\XgP8G\src\cpython\gil.jl:10 [inlined]
with_gil at ….julia\packages\PythonCall\XgP8G\src\cpython\gil.jl:9 [inlined]
init_context at ….julia\packages\PythonCall\XgP8G\src\cpython\context.jl:132
init at ….julia\packages\PythonCall\XgP8G\src\cpython\CPython.jl:21
jfptr___init___70514 at …\julia\ (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
jl_module_run_initializer at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:73
_finish_julia_init at /cygdrive/c/buildbot/worker/package_win64/build/src\init.c:796
jl_repl_entrypoint at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:695
mainCRTStartup at /cygdrive/c/buildbot/worker/package_win64/build/cli\loader_exe.c:42
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)

I’ve rencently got PyCall.jl portable by replacing the dll finding process with reading environment variables.

Conda.jl is harder to get work. I left it remain the wrong configs (local configs), but replacing all related variables with reading references which will be setup when __init__ PyCall module.

1 Like

PythonCall now supports this.

I made this small package for configuration.

Things will work smoothly after this PR gets merges


I made a similar PR to PyCall.jl. It relies on another package as well, CachePath.jl. It would be great to put together a doc on the pros and cons of various approaches, including startup times, etc. Many of the important questions are not documented (or not clearly), so it will take a bit of work.

1 Like

I’ve tried to add JuliaPythonAdaptor into this small package to test compilation, but it failed with the same error as in

The package has now run into some issue and fails at configuring when loaded as a precompiled package. If precompiled, JuliaPythonAdaptor is always loaded after PythonCall…

Install JuliaPythonAdaptor via pip and try

python -c "import JuliaPythonAdaptor;import os;del os.environ['JULIA_PYTHONCALL_LIBPTR'];os.system('path/to/you_app_exe')"

On the python side?
Path to which executable should I provide at os.system('path/to/you_exe'), python or julia?

The path to the executable you built with PackageCompiler.

Seems like python doesn’t recognize package name:

PS I:\> python -c "import JuliaPythonAdaptor;import os;del os.environ['JULIA_PYTHONCALL_LIBPTR'];os.system('C:\\_KTAuto\\backend\\conda\\3\\python.exe')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'JuliaPythonAdaptor'

Ok, now I try to install it with pip and get the following:

PS I:\> python -m pip install
     \ 183.2 kB 381.8 kB/s 0:00:00
  ERROR: Cannot unpack file C:\Users\gvg\AppData\Local\Temp\pip-unpack-u150j3_z\JuliaPythonAdaptor.jl (downloaded from C:\Users\gvg\AppData\Local\Temp\pip-req-build-wl7iqp0o, content-type: text/html; charset=utf-8); cannot detect archive format
ERROR: Cannot determine archive format of C:\Users\gvg\AppData\Local\Temp\pip-req-build-wl7iqp0o

Have you tried pip install JuliaPythonAdaptor?

Ok, I have installed it on python side and re-compiled app once more, but it keeps running with the same error. Should I change something in this particular script?

python -c "import JuliaPythonAdaptor;import os;del os.environ['JULIA_PYTHONCALL_LIBPTR'];os.system('C:\\_KTAuto\\backend\\conda\\3\\python.exe')"

julia --project=@. --startup-file=no -e '
ENV["JP_ADAPTOR_PY_EXE"] = "C:\\_KTAuto\\backend\\conda\\3\\python.exe";
using PackageCompiler;
PackageCompiler.create_app(pwd(), "PythonCallCompiled";

Or maybe there are some variables that I can check after importing Julia package?