I tried to compile a relocatable app with
PackageCompiler.create_app and call Python from it.
- With PyCall.jl - there is a problem that compiled binary has an absolute path to python library on dev machine (on my Windows machine it is
C:\Users\gvg\.julia\conda\3\python39.dll), so when I’m on another machine with different user - it can’t find the python by this path. See issue here: https://github.com/JuliaPy/PyCall.jl/issues/981
- With PythonCall.jl - there is another error after compilation, looks like it can’t find
Manifes.toml in compiled project files. See issue here: https://github.com/cjdoris/PythonCall.jl/issues/146
Any other ways I’ve missed?
With option 1, maybe I can set another python path before compilation, so it is not bound to a specific user?
I’m also finding PythonCall and PackageCompiler not working well together. It doesn’t use the JULIA_PYTHONCALL_EXE environment variable when compiled. For large sysimage with all packages included, it failed all together when starting julia with the compiled image:
fatal: error thrown and no exception handler available.
InitError(mod=:C, error=AssertionError(msg=“CTX.which == :PyCall”))
#34 at ….julia\packages\PythonCall\XgP8G\src\cpython\context.jl:135
with_gil at ….julia\packages\PythonCall\XgP8G\src\cpython\gil.jl:10 [inlined]
with_gil at ….julia\packages\PythonCall\XgP8G\src\cpython\gil.jl:9 [inlined]
init_context at ….julia\packages\PythonCall\XgP8G\src\cpython\context.jl:132
init at ….julia\packages\PythonCall\XgP8G\src\cpython\CPython.jl:21
jfptr___init___70514 at …\julia\mysysimage.so (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
jl_module_run_initializer at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:73
_finish_julia_init at /cygdrive/c/buildbot/worker/package_win64/build/src\init.c:796
jl_repl_entrypoint at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:695
mainCRTStartup at /cygdrive/c/buildbot/worker/package_win64/build/cli\loader_exe.c:42
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
I’ve rencently got PyCall.jl portable by replacing the dll finding process with reading environment variables.
Conda.jl is harder to get work. I left it remain the wrong configs (local configs), but replacing all related variables with reading references which will be setup when
__init__ PyCall module.
PythonCall now supports this.
I made this small package for configuration.
Things will work smoothly after this PR gets merges
I made a similar PR to PyCall.jl. It relies on another package as well, CachePath.jl. It would be great to put together a doc on the pros and cons of various approaches, including startup times, etc. Many of the important questions are not documented (or not clearly), so it will take a bit of work.
I’ve tried to add JuliaPythonAdaptor into this small package to test compilation, but it failed with the same error as in Readme.md:
The package has now run into some issue and fails at configuring when loaded as a precompiled package. If precompiled, JuliaPythonAdaptor is always loaded after PythonCall…
Install JuliaPythonAdaptor via pip and try
python -c "import JuliaPythonAdaptor;import os;del os.environ['JULIA_PYTHONCALL_LIBPTR'];os.system('path/to/you_app_exe')"
On the python side?
Path to which executable should I provide at
os.system('path/to/you_exe'), python or julia?
The path to the executable you built with PackageCompiler.
Seems like python doesn’t recognize package name:
PS I:\> python -c "import JuliaPythonAdaptor;import os;del os.environ['JULIA_PYTHONCALL_LIBPTR'];os.system('C:\\_KTAuto\\backend\\conda\\3\\python.exe')"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'JuliaPythonAdaptor'
Ok, now I try to install it with pip and get the following:
PS I:\> python -m pip install https://github.com/thautwarm/JuliaPythonAdaptor.jl
\ 183.2 kB 381.8 kB/s 0:00:00
ERROR: Cannot unpack file C:\Users\gvg\AppData\Local\Temp\pip-unpack-u150j3_z\JuliaPythonAdaptor.jl (downloaded from C:\Users\gvg\AppData\Local\Temp\pip-req-build-wl7iqp0o, content-type: text/html; charset=utf-8); cannot detect archive format
ERROR: Cannot determine archive format of C:\Users\gvg\AppData\Local\Temp\pip-req-build-wl7iqp0o
Have you tried
pip install JuliaPythonAdaptor?
Ok, I have installed it on python side and re-compiled app once more, but it keeps running with the same error. Should I change something in this particular script?
python -c "import JuliaPythonAdaptor;import os;del os.environ['JULIA_PYTHONCALL_LIBPTR'];os.system('C:\\_KTAuto\\backend\\conda\\3\\python.exe')"
julia --project=@. --startup-file=no -e '
ENV["JP_ADAPTOR_PY_EXE"] = "C:\\_KTAuto\\backend\\conda\\3\\python.exe";
Or maybe there are some variables that I can check after importing Julia package?