Hi,
I am building a julia library using create_library that I use in a larger python standalone app to be run on windows machines.
It works fine on several machine but when I try to relocate the app to a computer without julia installed (I use a fresh virtual windows 10 for testing)
I get an error when initializing the julia library from the python module.
It seems that julia cannot find the artifact “OpenSpecFun” but it is most likely the case with all artifacts.
My question is then how do you link the artifacts in this case?
I uses julia 1.7.3 with PackageCompiler 2.0.6
The error message is the following:
fatal: error thrown and no exception handler available.
InitError(mod=:OpenSpecFun_jll, error=ErrorException("Artifact "OpenSpecFun" was not installed correctly. Try `using Pkg; Pkg.instantiate()` to re-install all missing resources."))
error at .\error.jl:33
_artifact_str at C:\Users\StephaneBeaussier\AppData\Local\Programs\Julia-1.7.3\share\julia\stdlib\v1.7\Artifacts\src\Artifacts.jl:551
jfptr__artifact_str_44540 at C:\Users\stefa\AppData\Local\Programs\Python\Python37\lib\site-packages\vortex\lib\bin\Vortex.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
jl_f__call_latest at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:757
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
do_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:713
#invokelatest#2 at .\essentials.jl:716
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
do_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:713
invokelatest at .\essentials.jl:714
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
do_call at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:126
eval_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:215
eval_stmt_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:166 [inlined]
eval_body at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:587
jl_fptr_interpret_call at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:675
macro expansion at C:\Users\StephaneBeaussier\AppData\Local\Programs\Julia-1.7.3\share\julia\stdlib\v1.7\Artifacts\src\Artifacts.jl:680 [inlined]
find_artifact_dir at C:\Users\StephaneBeaussier\.julia\packages\JLLWrappers\QpMQW\src\wrapper_generators.jl:17
__init__ at C:\Users\StephaneBeaussier\.julia\packages\OpenSpecFun_jll\1Zaof\src\wrappers\x86_64-w64-mingw32-libgfortran5.jl:8
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
jl_module_run_initializer at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:73
_finish_julia_init at /cygdrive/c/buildbot/worker/package_win64/build/src\init.c:796
jl_init_with_image at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:74 [inlined]
jl_init_with_image at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:63
DllCanUnloadNow at C:\Users\stefa\AppData\Local\Programs\Python\Python37\DLLs\_ctypes.pyd (unknown line)
DllCanUnloadNow at C:\Users\stefa\AppData\Local\Programs\Python\Python37\DLLs\_ctypes.pyd (unknown line)
DllCanUnloadNow at C:\Users\stefa\AppData\Local\Programs\Python\Python37\DLLs\_ctypes.pyd (unknown line)
DllCanUnloadNow at C:\Users\stefa\AppData\Local\Programs\Python\Python37\DLLs\_ctypes.pyd (unknown line)
unknown function (ip: 00007ffa395f6a2b)
PyObject_FastCallKeywords at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyArg_UnpackStack at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalCodeWithName at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyOS_FSPath at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyOS_FSPath at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyMethodDef_RawFastCallDict at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalCodeWithName at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyArg_UnpackStack at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyArg_UnpackStack at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyArg_UnpackStack at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyArg_UnpackStack at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyFunction_FastCallDict at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyObject_CallMethodIdObjArgs at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyObject_CallMethodIdObjArgs at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyBytes_Fini at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyImport_ImportModuleLevelObject at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalCodeWithName at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyOS_FSPath at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyOS_FSPath at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyMethodDef_RawFastCallDict at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalCodeWithName at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyArg_UnpackStack at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyArg_UnpackStack at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyArg_UnpackStack at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyArg_UnpackStack at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyFunction_FastCallDict at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyObject_CallMethodIdObjArgs at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyObject_CallMethodIdObjArgs at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyBytes_Fini at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyImport_ImportModuleLevelObject at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalCodeWithName at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalCodeEx at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyEval_EvalCode at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyArena_Free at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyRun_FileExFlags at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyRun_SimpleFileExFlags at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyRun_AnyFileExFlags at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
Py_UnixMain at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
Py_UnixMain at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
PyErr_NoMemory at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
Py_Main at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
Py_Main at C:\Users\stefa\AppData\Local\Programs\Python\Python37\python37.dll (unknown line)
unknown function (ip: 00007ff66bbe126c)
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
It seems to me that that the problem rely to the artifacts directory link.
I have tried to add the artifact directory to the path within the python wrapper of my package without any success.
The best for my would be to be able to specify the path to the artifacts file from the python module initialisation script like I do for the .dll as shown below:
from collections import namedtuple
import os
import ctypes
import distutils.spawn
import platform
import numpy as np
import warnings
import multiprocessing
from ctypes import Structure, c_int8, c_char_p, c_int32
nthreads = multiprocessing.cpu_count()
class struct_jl_options(Structure):
_fields_ = [("quiet", c_int8),
("banner", c_int8),
("julia_bindir", c_char_p),
("julia_bin", c_char_p),
("cmds", c_char_p),
("image_file", c_char_p),
("cpu_target", c_char_p),
("nthreads", c_int32),
("nprocs", c_int32),
("machine_file", c_char_p),
("project", c_char_p),
("isinteractive", c_int8),
("color", c_int8),
("historyfile", c_int8),
("startupfile", c_int8),
("compile_enabled", c_int8),
("code_coverage", c_int8),
("malloc_log", c_int8),
("opt_level", c_int8),
("debug_level", c_int8),
("check_bounds", c_int8),
("depwarn", c_int8),
("warn_overwrite", c_int8),
("can_inline", c_int8),
("polly", c_int8),
("trace_compile", c_char_p),
("fast_math", c_int8),
("worker", c_int8),
("cookie", c_char_p),
("handle_signals", c_int8),
("use_sysimage_native_code", c_int8),
("use_compiled_modules", c_int8),
("bindto", c_char_p),
("outputbc", c_char_p),
("outputunoptbc", c_char_p),
("outputo", c_char_p),
("outputasm", c_char_p),
("outputji", c_char_p),
("output_code_coverage", c_char_p),
("incremental", c_int8),
("image_file_specified", c_int8),
("warn_scope", c_int8)]
if platform.system() == "Windows":
extension = "dll"
os.environ["PATH"] = os.path.abspath(os.path.join(os.path.dirname(__file__), "lib","bin")) + ";" + os.environ["PATH"]
os.environ["PATH"] = os.path.abspath(os.path.join(os.path.dirname(__file__), "lib","share","julia","artifacts","3e683ec5ca945a5aca74c49e8cccdf37c19b84a3","bin")) + ";" + os.environ["PATH"]
julialibpath = os.path.abspath(os.path.join(os.path.dirname(__file__), "lib","bin", f"libjulia.{extension}"))
libpath = os.path.abspath(os.path.join(os.path.dirname(__file__), "lib","bin", f"Vortex.{extension}"))
else:
pass
assert os.path.isfile(julialibpath)
julialib = ctypes.CDLL(julialibpath, ctypes.RTLD_GLOBAL)
jl_options = struct_jl_options.in_dll(julialib, "jl_options")
#jl_options.trace_compile = "stderr".encode("ascii")
jl_options.nthreads = 2 #nthreads
assert os.path.isfile(libpath)
mylib = ctypes.CDLL(libpath, ctypes.RTLD_GLOBAL)
try:
jl_init_with_image = julialib.jl_init_with_image
except AttributeError:
jl_init_with_image = julialib.jl_init_with_image__threading
jl_init_with_image.argtypes = [ctypes.c_char_p, ctypes.c_char_p]
jl_init_with_image(None, str(libpath).encode("utf-8"))
Also, here is the structure of my python module that call the julia dll
vortex
\tree
|-- __init__.py
|-- lib.py (initialize julia)
|-- lib
| |-- bin
| | |-- all the julia dll
| |-- share
|-- julia
|-- artifacts
|-- all the artifacts directory