RAM usage of building a system image

My script that is building a system image needs about 50 GB of RAM. This is very expensive. I am trying to use a swap file, but this is getting very slow.
See for example: Install KiteModels#main · OpenSourceAWE/AWEMeta.jl@2a4ca15 · GitHub

Did not finish in 4h, now trying 6h.

Any ideas how to reduce the RAM usage? I switched from ControlPlots.jl to MakieControlPlots.jl. Before it was kind of OK.

You can find the script that builds the system image here:

Do you think using JULIA_HEAP_SIZE_HINT could help?

You could check out aotcompile: Release the Julia heap before compiling (let's use all the threads) - Pull Request #63531 - JuliaLang/julia - GitHub

Nice that things will improve with Julia 1.14, but why this large regression from Julia 1.12 to Julia 1.13: Building a system image using Julia 1.13 fails (due to excessive RAM usage) · Issue #63141 · JuliaLang/julia · GitHub

Can’t this regression be adressed?

A closer investigation revealed that it is not a Julia regression, but a PackageCompiler regression:

Since #1096 (released in PackageCompiler 2.4.1), create_sysimage defaults to cpu_target="sysimage" on Julia 1.13+, while older Julia versions keep "native". With the official Julia binaries this expands to

generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1);x86-64-v4,-rdrnd,base(1)

so every function is compiled at least twice, plus partial clones for two more targets. For large sysimages this makes the “compiling incremental system image” step need more than twice the memory and several times as long.

Workaround: Use cpu_target="native". This fixed the issue for me.

New issue in PackageCompiler: Default `cpu_target="sysimage"` on Julia 1.13 more than doubles sysimage build memory and time · Issue #1143 · JuliaLang/PackageCompiler.jl · GitHub