LoadError with PackageCompiler with Docker on macOS (works in Docker on Windows, Docker on Linux, and in native macOS)

General Usage
1

When I’m running the PackageCompiler to generate a Julia system image, I’m getting a LoadError. What’s so odd is that this exact process works in Docker on Windows, in Docker on Linux, on macOS (native), but not in Docker on macOS – the only place I see the error. Unfortunately, I need this to run everywhere.

First, here’s the error:

[ Info: PackageCompiler: creating system image object file, this might take a while...
ERROR: LoadError: failed process: Process(`/opt/julia-1.4.1/bin/julia --color=yes --startup-file=no --cpu-target=native --sysimage=/opt/julia-1.4.1/lib/julia/sys.so --project=/Systems/build/host_linux/sim/PlantCompiler --output-o=/tmp/jl_vDTTVa.o -e 'Base.reinit_stdio()
@eval Sys BINDIR = ccall(:jl_get_julia_bindir, Any, ())::String
Base.init_load_path()
Base.init_depot_path()
using Plant
# This @eval prevents symbols from being put into Main
@eval Module() begin
    PrecompileStagingArea = Module()
    for (_pkgid, _mod) in Base.loaded_modules
        if !(_pkgid.name in ("Main", "Core", "Base"))
            eval(PrecompileStagingArea, :(const $(Symbol(_mod)) = $_mod))
        end
    end
    precompile_statements = String[]
        append!(precompile_statements, readlines("/tmp/jl_Cywj8I"))

    for statement in sort(precompile_statements)
        # println(statement)
        try
            Base.include_string(PrecompileStagingArea, statement)
        catch
            # See julia issue #28808
            @debug "failed to execute $statement"
        end
    end
end # module
empty!(LOAD_PATH)
empty!(DEPOT_PATH)
'`, ProcessSignaled(9)) [0]

Stacktrace:
 [1] pipeline_error at ./process.jl:525 [inlined]
 [2] run(::Cmd; wait::Bool) at ./process.jl:440
 [3] run at ./process.jl:438 [inlined]
 [4] create_sysimg_object_file(::String, ::Array{String,1}; project::String, base_sysimage::String, precompile_execution_file::Array{String,1}, precompile_statements_file::Array{String,1}, cpu_target::String, script::Nothing, isapp::Bool) at /root/.julia/packages/PackageCompiler/xtztx/src/PackageCompiler.jl:292
 [5] create_sysimage(::Symbol; sysimage_path::String, project::String, precompile_execution_file::String, precompile_statements_file::Array{String,1}, incremental::Bool, filter_stdlibs::Bool, replace_default::Bool, cpu_target::String, script::Nothing, base_sysimage::Nothing, isapp::Bool) at /root/.julia/packages/PackageCompiler/xtztx/src/PackageCompiler.jl:423
 [6] compile_plant() at /Systems/build/host_linux/sim/PlantCompiler/src/PlantCompiler.jl:28
 [7] top-level scope at /Systems/build/host_linux/sim/PlantCompiler/src/plant_compiler_script.jl:9
 [8] include(::Module, ::String) at ./Base.jl:377
 [9] exec_options(::Base.JLOptions) at ./client.jl:288
 [10] _start() at ./client.jl:484
in expression starting at /Systems/build/host_linux/sim/PlantCompiler/src/plant_compiler_script.jl:9

I’m working on making a small example to try to isolate the behavior. I don’t have that yet.
I can build the example (https://julialang.github.io/PackageCompiler.jl/dev/sysimages/) and call it the same was I call the PackageCompiler for my own application, and that works.

My next move will be to start commenting things out of the driver (precompile script), and then commenting things out of the modules I use to try to binary search for the problem.

Does anyone have any tips for where to look?

1 Like
2

Try increasing the memory (RAM) that Docker Desktop has. The default is 2 GB of memory. Try setting it to 4 GB, or even higher.

5 Likes
3

Nailed it, @dilumaluthge! I spent all day debugging things related to this. I did not expect Julia+PackageCompiler to be using so much RAM. Thank you so much!

I’ll go look for ways to warn users about this, which I assume is going to be a Docker thing and not a Julia/PackageCompiler thing.

4

It could be good to add a message to the PackageCompiler documentation.

1 Like
5

Yeah, and I’d sign up to help with that PR, but how would the PackageCompiler know? I’m a bit out of my element with stuff like that.

1 Like
6

Doesn’t have to be in the code for PackageCompiler.

Just add a note to the documentation saying something like “if you see this error when using PackageCompiler inside Docker, try increasing the RAM.”

@kristoffer.carlsson

1 Like