Application does not load all artifacts during compilation

I’m currently trying to compile an app using PackageCompiler. The associated project App depends on two packages Foo and Bar.
Foo itself depends on MKL which depends on MKL_jll (which are present in the Manifest.toml of App)

My current compilation is performed using

create_app(<package_dir>,<exebinpath>; precompile_execution_file=<pspath>,incremental = true, cpu_target="generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)", include_lazy_artifacts=true, include_transitive_dependencies = true)

Where the different path point to the correct targets.
Given that include_lazy artifacts = true and include_transitive_dependencies = true I’d thought that the executable produced by the compilation would have been portable (to a machine with the same OS and architecture).
However, whenever I try to run the application on the other machine, the first thing julia tries to do is to download MKL. The problem is that this machine cannot be connected to internet so it fails on the download.

It there a way to ensure that all dependencies / artifacts are loaded ? Why does include_lazy_artifacts not handle this ?

I’m working on this configuration

Julia Version 1.7.3
Commit 742b9abb4d (2022-05-06 12:58 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: 11th Gen Intel(R) Core™ i7-1185G7 @ 3.00GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-12.0.1 (ORCJIT, tigerlake)
Environment:
JULIA_EDITOR = code
JULIA_NUM_THREADS = 1

1 Like

Sounds like PackageCompiler does not download artifacts for MKL now · Issue #691 · JuliaLang/PackageCompiler.jl · GitHub. I’ll look at it.

1 Like

I just tested with v2.0.5 of PackageCompiler and the app doesn’t try to download artifacts, and weighs much more ( MKL ! ).

Just for the record, I asked a somewhat related question here if you have a suggestion. I preferred to split it in order to be able to focus on the two subjects separately.

Side note, If I try to create a sysimage rather than an app I get this error (with 2.0.5 and 2.0.6).
Is it due to the general non-relocatability of the sysimages or is is becaus the HDF5 artifact is not correctly handled ?

D:\admin\Desktop>julia -J sysimage.dll
fatal: error thrown and no exception handler available.
InitError(mod=:HDF5_jll, error=ErrorException(“Artifact “HDF5” was not installed correctly. Try using Pkg; Pkg.instantiate() to re-install all missing resources.”))
error at .\error.jl:33
_artifact_str at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\Artifacts\src\Artifacts.jl:551
jfptr__artifact_str_84373 at D:\admin\Desktop\sysimage.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
jl_f__call_latest at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:757
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
do_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:713
#invokelatest#2 at .\essentials.jl:716
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
do_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:713
invokelatest at .\essentials.jl:714
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
do_call at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:126
eval_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:215
eval_stmt_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:166 [inlined]
eval_body at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:587
jl_fptr_interpret_call at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:675
macro expansion at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\Artifacts\src\Artifacts.jl:680 [inlined]
find_artifact_dir at D:\admin.julia\packages\JLLWrappers\QpMQW\src\wrapper_generators.jl:17
init at D:\admin.julia\packages\HDF5_jll\5jwlp\src\wrappers\x86_64-w64-mingw32.jl:11
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
jl_module_run_initializer at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:73
_finish_julia_init at /cygdrive/c/buildbot/worker/package_win64/build/src\init.c:796
jl_repl_entrypoint at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:695
mainCRTStartup at /cygdrive/c/buildbot/worker/package_win64/build/cli\loader_exe.c:42
BaseThreadInitThunk at C:\windows\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\windows\SYSTEM32\ntdll.dll (unknown line)