I can not create “app” with PackageCompiler even for simplest GPU optimization with ExaModels+MadNLPGPU running cuDSS. I’ve made full MWE project with package to be built in attempt to help this to debug. Compiling does not pass with either LTS or release. It does not pass with compile=all command. Different error message appears if run to compile with --compile=all --strip-metadata --strip-ir, filter_stdlibs=true (maybe more meaningful than the one below; just swap two commands in setup.jl). Without compile=all, strip and filter commands, the compile passes but fails at runtime (calls multiple instances of julia.exe until it consumes all free RAM).
I attach MWE with toml’s for Julia 1.10.8 and 1.11.3. To run just set root dir as work dir and run setup.jl . I think this would be a big deal if this is solved. We need some kind of a test for this IMO.
mwe.7z on mega.nz (Uploading compressed folders on discourse is not allowed so I uploaded at free host.)
I am tagging some people that could be interested: @maleadt , @sshin23, @frapac, @amontoison, @odow, @sdanisch. Please ignore if you feel inappropriately tagged. I tagged mostly leaders of relevant packages. (Edit(odow): I’ve removed the explicit tags.)
Error log using Julia 1.10.8 and compile=all.
⠸ [05m:46s] PackageCompiler: compiling nonincremental system imageimmarg operand has non-immediate parameter
⢰ [05m:46s] PackageCompiler: compiling nonincremental system image %value_phi = phi i8 [ %guard_res8, %guard_exit6 ], [ %guard_res15, %guard_exit13 ]
immarg operand has non-immediate parameter
%value_phi = phi i32 [ %guard_res5, %guard_exit4 ], [ %guard_res12, %guard_exit10 ]
call void @llvm.nvvm.cp.async.wait.group(i32 %value_phi) [ “jl_roots”({} addrs⣠ [05m:46s] PackageCompiler: compiling nonincremental system image%28 = call <2 x i64> @llvm.x86.aesni.aeskeygenassist(<2 x i64> %unbox, i8 zeroext %value_phi) [ “jl_roots”({} addrspace(10)* %44) ], !dbg !125496
⣄ [05m:47s] PackageCompiler: compiling nonincremental system image%16) ], !dbg !187547
⠙ [05m:47s] PackageCompiler: compiling nonincremental system imageLLVM ERROR: Broken module found, compilation aborted![11476] signal (6.-6): Aborted
in expression starting at none:0
Allocations: 1520472196 (Pool: 1519649926; Big: 822270); GC: 645
[05m:48s] PackageCompiler: compiling nonincremental system image
ERROR: LoadError: failed process: Process(/home/karlo/.julia/juliaup/julia-1.10.8+0.x64.linux.gnu/bin/julia --color=yes --startup-file=no --pkgimages=no '--cpu-target=generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)' --compile=all --sysimage=/tmp/jl_eAnXJI/sys.so --project=/home/karlo/Desktop/mwe/gpu_opt.jl --output-o=/tmp/jl_EbIb0NGBid-o.a /tmp/jl_EN2JafOnKl, ProcessSignaled(6)) [0]Stacktrace:
[1] pipeline_error
@ ./process.jl:565 [inlined]
[2] run(::Cmd; wait::Bool)
@ Base ./process.jl:480
[3] run
@ ./process.jl:477 [inlined]
[4] #20
@ ~/.julia/packages/PackageCompiler/UbaS4/ext/TerminalSpinners.jl:157 [inlined]
[5] spin(f::PackageCompiler.var"#20#22"{Cmd}, s::PackageCompiler.TerminalSpinners.Spinner{Base.TTY})
@ PackageCompiler.TerminalSpinners ~/.julia/packages/PackageCompiler/UbaS4/ext/TerminalSpinners.jl:164
[6] macro expansion
@ ~/.julia/packages/PackageCompiler/UbaS4/ext/TerminalSpinners.jl:157 [inlined]
[7] create_sysimg_object_file(object_file::String, packages::Vector{String}, packages_sysimg::Set{Base.PkgId}; project::String, base_sysimage::String, precompile_execution_file::Vector{String}, precompile_statements_file::Vector{String}, cpu_target::String, script::Nothing, sysimage_build_args::Cmd, extra_precompiles::String, incremental::Bool, import_into_main::Bool)
@ PackageCompiler ~/.julia/packages/PackageCompiler/UbaS4/src/PackageCompiler.jl:130
[8] create_sysimg_object_file
@ ~/.julia/packages/PackageCompiler/UbaS4/src/PackageCompiler.jl:319 [inlined]
[9] create_sysimage(packages::Vector{String}; sysimage_path::String, project::String, precompile_execution_file::String, precompile_statements_file::Vector{String}, incremental::Bool, filter_stdlibs::Bool, cpu_target::String, script::Nothing, sysimage_build_args::Cmd, include_transitive_dependencies::Bool, base_sysimage::Nothing, julia_init_c_file::Nothing, julia_init_h_file::Nothing, version::Nothing, soname::Nothing, compat_level::String, extra_precompiles::String, import_into_main::Bool)
@ PackageCompiler ~/.julia/packages/PackageCompiler/UbaS4/src/PackageCompiler.jl:639
[10] create_sysimage
@ ~/.julia/packages/PackageCompiler/UbaS4/src/PackageCompiler.jl:527 [inlined]
[11] create_app(package_dir::String, app_dir::String; executables::Nothing, precompile_execution_file::String, precompile_statements_file::Vector{String}, incremental::Bool, filter_stdlibs::Bool, force::Bool, c_driver_program::String, cpu_target::String, include_lazy_artifacts::Bool, sysimage_build_args::Cmd, include_transitive_dependencies::Bool, include_preferences::Bool, script::Nothing)
@ PackageCompiler ~/.julia/packages/PackageCompiler/UbaS4/src/PackageCompiler.jl:886
[12] top-level scope
@ ~/Desktop/mwe/setup.jl:9
versioninfo()
Julia Version 1.10.8
Commit 4c16ff44be8 (2025-01-22 10:06 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: 32 × AMD Ryzen 9 5950X 16-Core Processor
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 32 virtual cores)
julia> MadNLPGPU.CUDA.versioninfo()
CUDA runtime 12.6, artifact installation
CUDA driver 12.4
NVIDIA driver 550.120.0CUDA libraries:
- CUBLAS: 12.6.4
- CURAND: 10.3.7
- CUFFT: 11.3.0
- CUSOLVER: 11.7.1
- CUSPARSE: 12.5.4
- CUPTI: 2024.3.2 (API 24.0.0)
- NVML: 12.0.0+550.120
Julia packages:
- CUDA: 5.6.1
- CUDA_Driver_jll: 0.10.4+0
- CUDA_Runtime_jll: 0.15.5+0
Toolchain:
- Julia: 1.10.8
- LLVM: 15.0.7
1 device:
0: NVIDIA GeForce RTX 2070 (sm_75, 7.388 GiB / 8.000 GiB available)