PackageCompiler create_sysimage fails because of failure to precompile Cxx.jl

Hello,

I am watching the awesome talk by Kristoffer Carlsson here PackageCompiler and Static Compilation - YouTube about PackageCompiler.

I tried to run the following command in my Julia repl create_sysimage([:DataFrames]; sysimage_path="customsysimage.dylib"). However, the command fails because it fails to precompile Cxx. Here is the full error. Can someone please help me fix this error? I tried reinstalling Cxx as well as Cxx#master. Tried to also build it using Pkg.build. But none of that helped.

Thank you.

Precompiling project...
  ✗ Cxx
  0 dependencies successfully precompiled in 3 seconds. 515 already precompiled.

ERROR: The following 1 direct dependency failed to precompile:

Cxx [a0b5b9ef-44b7-5148-a2d1-f6db19f3c3d2]

Failed to precompile Cxx [a0b5b9ef-44b7-5148-a2d1-f6db19f3c3d2] to /Users/curiouslearn/.julia/compiled/v1.8/Cxx/jl_DvckzR.
ERROR: LoadError: could not load library "/Users/curiouslearn/.julia/packages/Cxx/GNqHb/src/../deps/usr/lib/libcxxffi"
dlopen(/Users/curiouslearn/.julia/packages/Cxx/GNqHb/src/../deps/usr/lib/libcxxffi.dylib, 0x0009): Symbol not found: _jl_LLVMContext
  Referenced from: /Users/curiouslearn/.julia/packages/Cxx/GNqHb/deps/usr/lib/libcxxffi.dylib
  Expected in: /Applications/Julia-1.8.app/Contents/Resources/julia/lib/libjulia.1.8.dylib
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:117
  [2] dlopen
    @ ./libdl.jl:116 [inlined]
  [3] init_libcxxffi()
    @ Cxx.CxxCore ~/.julia/packages/Cxx/GNqHb/src/initialization.jl:23
  [4] top-level scope
    @ ~/.julia/packages/Cxx/GNqHb/src/initialization.jl:25
  [5] include(mod::Module, _path::String)
    @ Base ./Base.jl:419
  [6] include(x::String)
    @ Cxx.CxxCore ~/.julia/packages/Cxx/GNqHb/src/Cxx.jl:144
  [7] top-level scope
    @ ~/.julia/packages/Cxx/GNqHb/src/Cxx.jl:170
  [8] include
    @ ./Base.jl:419 [inlined]
  [9] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1554
 [10] top-level scope
    @ stdin:1
in expression starting at /Users/curiouslearn/.julia/packages/Cxx/GNqHb/src/initialization.jl:25
in expression starting at /Users/curiouslearn/.julia/packages/Cxx/GNqHb/src/Cxx.jl:142
in expression starting at stdin:1
Stacktrace:
 [1] pkgerror(msg::String)
   @ Pkg.Types /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/Pkg/src/Types.jl:67
 [2] precompile(ctx::Pkg.Types.Context, pkgs::Vector{String}; internal_call::Bool, strict::Bool, warn_loaded::Bool, already_instantiated::Bool, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ Pkg.API /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/Pkg/src/API.jl:1435
 [3] precompile
   @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/Pkg/src/API.jl:1072 [inlined]
 [4] #precompile#225
   @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/Pkg/src/API.jl:1071 [inlined]
 [5] precompile (repeats 2 times)
   @ /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/stdlib/v1.8/Pkg/src/API.jl:1071 [inlined]
 [6] top-level scope
   @ none:1
ERROR: failed process:
Process(setenv(`/Applications/Julia-1.8.app/Contents/Resources/julia/bin/julia
--color=yes --startup-file=no
--sysimage=/Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/sys.dylib
-e 'using Pkg; Pkg.precompile()'`,["XPC_FLAGS=0x0", "COMMAND_MODE=unix2003",
"EDITOR=/usr/local/bin/nvim",
"PWD=/Users/curiouslearn/Dropbox",
"PATH=/Users/curiouslearn/.opam/cs3110/bin:/Users/curiouslearn/.cabal/bin:/usr/local/opt/openjdk/bin:/Users/curiouslearn/.opam/cs3110/bin:/usr/local/opt/grep/libexec/gnubin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/munki:/Applications/Privileges.app/Contents/Resources:/Library/TeX/texbin:/Applications/Julia-1.8.app/Contents/Resources/julia/bin/:/Users/curiouslearn/bin:/Users/curiouslearn/go/bin:/Users/curiouslearn/.cargo/bin:/Users/curiouslearn/anaconda3/bin:/Applications/Racket
v8.0/bin:/Users/curiouslearn/Library/Application
Support/Coursier/bin:/Users/curiouslearn/.emacs.d/bin:/Users/curiouslearn/.nvm/versions/node/v14.15.5/bin/:/Users/curiouslearn/.opam/default/bin:/Users/curiouslearn/.local/share/ponyup/bin:/Users/curiouslearn/.ghcup/bin",
"XPC_SERVICE_NAME=0", "TERM_PROGRAM=iTerm.app",
"JULIA_LOAD_PATH=/Users/curiouslearn/.julia/environments/v1.8:@stdlib",
"HOME=/Users/curiouslearn", "TERM=xterm-256color",
"TERM_PROGRAM_VERSION=3.4.19", "COLORTERM=truecolor"]), ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error
   @ ./process.jl:565 [inlined]
 [2] run(::Cmd; wait::Bool)
   @ Base ./process.jl:480
 [3] run
   @ ./process.jl:477 [inlined]
 [4] ensurecompiled(project::String, packages::Vector{String}, sysimage::String)
   @ PackageCompiler ~/.julia/packages/PackageCompiler/i90pX/src/PackageCompiler.jl:282
 [5] create_sysimage(packages::Vector{Symbol}; sysimage_path::String, project::String, precompile_execution_file::Vector{String}, precompile_statements_file::Vector{String}, incremental::Bool, filter_stdlibs::Bool, cpu_target::String, script::Nothing, sysimage_build_args::Cmd, include_transitive_dependencies::Bool, base_sysimage::Nothing, julia_init_c_file::Nothing, version::Nothing, soname::Nothing, compat_level::String, extra_precompiles::String)
   @ PackageCompiler ~/.julia/packages/PackageCompiler/i90pX/src/PackageCompiler.jl:533
 [6] top-level scope
   @ REPL[4]:1

If anyone has similar problems, I was able to fix it by removing Cxx using Pkg.rm("Cxx"). Now I need to figure out which package installed it, and relies on it. I do not remember installing it except today. I tried to install/update it today because, PackageCompiler was not able to create a sysimage because of the Cxx precompilation problem.

The Manifest should show where the dependency was introduced under the deps listing. Cxx.jl is a great package, but since it is tightly coupled to LLVM, it has not been supported since Julia 1.3: GitHub - JuliaInterop/Cxx.jl: The Julia C++ Interface

Thank you @sjkelly. I opened and searched through Manifest.toml, but did not find any occurrence of Cxx. I must have installed it sometime in the past, but don’t remember. At least, it is good to know that none of my current packages depend on it (since it does not appear in Manifest.toml).

1 Like