Binaries made by PackageCompiler.jl hangs while maxing out RAM

General Usage
1

Linux binaries (WSL Ubuntu) made by PackageCompiler.jl hangs (>1hr) while maxing out RAM. Windows binaries work though. How to debug?

2

I guess you need to tell us more about the code that you compiled to get a useful answer.

3

It’s a FDTD simulator GitHub - paulxshen/Luminescent.jl

Windows binaries work.
Linux binaries hangs now but worked few months ago.
Tried couple different Linux machines to same effect.
My entry point returns immediately on empty ARGS - still hangs in this case - so it’s not even executing my code.

4

I guess I can try using older PackageCompiler when it last worked. But still nice to know what’s causing the current issue

5

Can you:

  1. provide the output of using Pkg; Pkg.status() ?
  2. provide the code you use to run PackageCompiler ?
6

Thanks, here:

[fe5ef43c] ArrayPadding v1.0.8
[ab4f0b2a] BFloat16s v0.5.0
[052768ef] CUDA v5.6.0
[13f3f980] CairoMakie v0.12.18
[d360d2e6] ChainRulesCore v1.25.1
[864edb3b] DataStructures v0.18.20
[5789e2e9] FileIO v1.16.6
[587475ba] Flux v0.16.0
[d9f16b24] Functors v0.5.2
[c817782e] ImageBase v0.1.7
[682c06a0] JSON v0.21.4
[6872b481] Jello v1.0.11
[15e1cf62] NPZ v0.4.3
[3bd65402] Optimisers v0.4.4
[2ae35dd2] Permutations v0.4.22
[3b53a3d7] Porcupine v1.0.10
[10745b16] Statistics v1.11.1
[3a884ed6] UnPack v1.0.2
⌅ [e88e6eb3] Zygote v0.6.75
[ade2ca70] Dates v1.11.0
[37e2e46d] LinearAlgebra v1.11.0
[9a3f8284] Random v1.11.0
[2f01184e] SparseArrays v1.11.0

using PackageCompiler
ENV[“JULIA_SSL_CA_ROOTS_PATH”] = “”

create_app(“.”, “…/LuminescentAI”,
executables=[
“lumi” => “lumi”,
],
precompile_execution_file=“build/precompile_app.jl”,
force=true
)

7

Tried incremental=false didn’t work. Tried rolling back to 2.1.17 but got some jll error on julia 1.11 - not worth debugging.

Odd why compiling on windows works…

8

One thing I had to do was to add include_transitive_dependencies=true,:

PackageCompiler.create_sysimage(
    [:Dierckx, :StaticArrays, :Parameters, :NLsolve, :DocStringExtensions, :Sundials, :KiteUtils, 
     :KitePodModels, :AtmosphericModels, :OrdinaryDiffEqCore, :OrdinaryDiffEqBDF, :OrdinaryDiffEqSDIRK,
     :ModelingToolkit, :DSP, :JLD2, :Colors];
    sysimage_path="kps-image_tmp.so",
    include_transitive_dependencies=true,
    precompile_execution_file=joinpath("test", "test_for_precompile.jl")
)

Furthermore, if you want more support I am afraid you have to provide a “minimal working example” that allows others to reproduce your problem, see: Please read: make it easier to help you

9

Thanks I think include_transitive_dependencies=true is default - anyways I tried it and didn’t work. Also tried w/o luck incremental true. MWE is good practice but IDK if it has any additional useful info here. Meanwhile I interrupted ^C when it hangs and got:

···
InterruptException()
_jl_mutex_unlock at /cache/build/tester-amdci5-12/julialang/julia-release-1-dot-11/src/threading.c:949
jl_mutex_unlock at /cache/build/tester-amdci5-12/julialang/julia-release-1-dot-11/src/julia_locks.h:80 [inlined]
ijl_task_get_next at /cache/build/tester-amdci5-12/julialang/julia-release-1-dot-11/src/scheduler.c:526
^C^Cpoptask at ./task.jl:1012
wait at ./task.jl:1021
task_done_hook at ./task.jl:694
jfptr_task_done_hook_378591.1 at /home/ubuntu/LuminescentAI/lib/julia/sys.so (unknown line)
jl_apply at /cache/build/tester-amdci5-12/julialang/julia-release-1-dot-11/src/julia.h:2157 [inlined]
jl_finish_task at /cache/build/tester-amdci5-12/julialang/julia-release-1-dot-11/src/task.c:319
start_task at /cache/build/tester-amdci5-12/julialang/julia-release-1-dot-11/src/task.c:1213
···

Because it’s hanging even on a dummy function that returns 0, I assume it’s stuck deserializing the sysimage? There were couple other github issues on apps hanging but no responses. .

10

(post deleted by author)

11

See also: Compiled app hangs in Ubuntu · Issue #1025 · JuliaLang/PackageCompiler.jl · GitHub

Did you try if it works with Julia 1.10?

12

Thanks I’ll just answer your question here:

Julia Version 1.11.2
Commit 5e9a32e7af2 (2024-12-01 20:02 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: 16 × AMD EPYC 7R32
WORD_SIZE: 64
LLVM: libLLVM-16.0.6 (ORCJIT, znver2)
Threads: 1 default, 0 interactive, 1 GC (on 16 virtual cores)

Also failed on another machine on WSL
I tried PackageCompiler (latest vs 2.1.17) on julia 1.10.6 - still didn’t work

13

Thanks for trying Julia 1.10.6 !

Now we only need a reproducable MWE (minimal working example) so that someone else can reproduce your problem.