Complex workaround needed. Why?

I need this workaround to avoid crashes due to version conflicts od shared libraries on Linux:

if [[ "$(uname -s)" == "Linux" ]]; then
    export FONTCONFIG_FILE=/etc/fonts/fonts.conf
    # On Ubuntu 22.04 the system has: OpenSSL 3.0.2 (too old), fontconfig 2.13 (missing
    # FcConfigSetDefaultSubstitute). The dynamic linker reuses already-loaded system libs
    # instead of the bundled ones, so we force the bundled versions in with LD_PRELOAD.
    # libiconv/libintl are absent on the system (glibc-internal on Ubuntu), so we add
    # their artifact dirs to LD_LIBRARY_PATH so dependency resolution finds them without
    # hitting circular-preload ordering problems.
    _PRELOADS=()
    _LIBDIRS=()

    # Get all Julia depot paths
    _DEPOT_PATHS=()
    if [[ -n "${JULIA_DEPOT_PATH:-}" ]]; then
        IFS=':' read -ra _DEPOT_PATHS <<< "$JULIA_DEPOT_PATH"
    else
        _DEPOT_PATHS=("$HOME/.julia")
    fi

    # libcrypto: override stale system version via LD_PRELOAD
    _BUNDLED=""
    for _depot in "${_DEPOT_PATHS[@]}"; do
        _BUNDLED=$(find "$_depot/artifacts" -maxdepth 3 -name "libcrypto.so.3" -path "*/lib/*" 2>/dev/null | head -1)
        [[ -n "$_BUNDLED" ]] && break
    done
    [[ -n "$_BUNDLED" ]] && _PRELOADS+=("$_BUNDLED")

    # libiconv + libintl: absent on Ubuntu 22.04, add dirs to LD_LIBRARY_PATH so
    # circular dep between them resolves naturally without preload ordering issues.
    for _libname in libiconv.so.2 libintl.so.8; do
        _BUNDLED=""
        for _depot in "${_DEPOT_PATHS[@]}"; do
            _BUNDLED=$(find "$_depot/artifacts" -maxdepth 3 -name "$_libname" -path "*/lib/*" 2>/dev/null | head -1)
            [[ -n "$_BUNDLED" ]] && break
        done
        if [[ -n "$_BUNDLED" ]]; then
            _dir=$(dirname "$_BUNDLED")
            [[ ":${_LIBDIRS[*]}:" != *":$_dir:"* ]] && _LIBDIRS+=("$_dir")
        fi
    done

    # libfontconfig: override stale system version via LD_PRELOAD; also add its dir to
    # LD_LIBRARY_PATH so its deps (libiconv etc.) are found during preload resolution.
    # Pick the version that exports FcConfigSetDefaultSubstitute (needed by Pango β‰₯ 1.57).
    _BUNDLED=""
    for _depot in "${_DEPOT_PATHS[@]}"; do
        for _fc in $(find "$_depot/artifacts" -maxdepth 3 -name "libfontconfig.so.1" -path "*/lib/*" 2>/dev/null); do
            if nm -D "$_fc" 2>/dev/null | grep -q FcConfigSetDefaultSubstitute; then
                _BUNDLED="$_fc"
                break 2
            fi
        done
    done
    if [[ -n "$_BUNDLED" ]]; then
        _dir=$(dirname "$_BUNDLED")
        [[ ":${_LIBDIRS[*]}:" != *":$_dir:"* ]] && _LIBDIRS+=("$_dir")
        _PRELOADS+=("$_BUNDLED")
    fi

    [[ ${#_PRELOADS[@]} -gt 0 ]] && export LD_PRELOAD=$(IFS=:; echo "${_PRELOADS[*]}")
    if [[ ${#_LIBDIRS[@]} -gt 0 ]]; then
        _extra=$(IFS=:; echo "${_LIBDIRS[*]}")
        export LD_LIBRARY_PATH="${_extra}${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
    fi
    unset _PRELOADS _LIBDIRS _BUNDLED _DEPOT_PATHS _depot _fc _dir _extra _libname
fi
echo "LD_PRELOAD was: $LD_PRELOAD"
echo "LD_LIBRARY_PATH was: $LD_LIBRARY_PATH"

# On Linux, wrap julia to also preload the bundled libglib for Julia processes only.
# The system libglib is often too old for GLMakie/Glib_jll (missing g_string_copy).
# Scoping this to Julia calls prevents breaking system commands (uname, git, sed)
# which would fail if artifact libglib's transitive deps (libintl) are not satisfied.
if [[ "$(uname -s)" == "Linux" ]]; then
    _JULIA_GLIB_PRELOADS=""
    for _depot in $(echo "${JULIA_DEPOT_PATH:-$HOME/.julia}" | tr ':' ' '); do
        _f=$(find "$_depot/artifacts" -maxdepth 3 -name "libglib-2.0.so.0" -path "*/lib/*" 2>/dev/null | head -1)
        if [[ -n "$_f" ]]; then
            _glib_dir=$(dirname "$_f")
            _intl=""
            [[ -f "$_glib_dir/libintl.so.8" ]] && _intl="$_glib_dir/libintl.so.8:"
            _JULIA_GLIB_PRELOADS="${_intl}$_f"
            break
        fi
    done
    unset _depot _f _glib_dir _intl
    if [[ -n "$_JULIA_GLIB_PRELOADS" ]]; then
        julia() {
            LD_PRELOAD="${LD_PRELOAD:+$LD_PRELOAD:}$_JULIA_GLIB_PRELOADS" command julia "$@"
        }
    fi
fi

Why is this needed for KiteModels.jl?

Possible reasons:

  • conflicts between Conda, PyPlot and Makie
  • differences between CI and local machine
  • the use of PackageCompiler

Still, I think there must be something very wrong if I need such a workaround. Any ideas what?

Might be clearer if you explained the issue you’re trying to solve, rather than the overcomplicated workaround you (?) came up with.

You can see what happens without this workaround in the failed CI runs here: here

SciMLBase β†’ SciMLBaseMakieExt 

Failed to precompile SciMLBaseMakieExt [565f26a4-c902-5eae-92ad-e10714a9d9de] to "/tmp/testdepot/compiled/v1.12/SciMLBaseMakieExt/jl_SBQvyJ".
ERROR: LoadError: InitError: could not load library "/tmp/testdepot/artifacts/654863f618f22eeb74ce2d1b4996666e65e596e4/lib/libgio-2.0.so"
/tmp/testdepot/artifacts/654863f618f22eeb74ce2d1b4996666e65e596e4/lib/libgobject-2.0.so.0: undefined symbol: g_string_copy
Stacktrace:
  [1] #dlopen#3
    @ ./libdl.jl:120 [inlined]
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:119
  [3] macro expansion

and here:


β”Œ KiteViewers
β”‚  β”Œ Warning: Could not find font /usr/share/fonts/truetype/freefont/FreeMono.ttf, using TeX Gyre Heros Makie
β”‚  β”” @ Makie /tmp/testdepot/packages/Makie/Vn16E/src/conversions.jl:1456
β””  
[ Info: PackageCompiler: Executing /tmp/test/KiteModels.jl/test/test_for_precompile.jl => /tmp/jl_packagecompiler_C8f6P5/jl_I0qAVG
ERROR: LoadError: InitError: could not load library "/tmp/testdepot/artifacts/b2e490a224990d92e99cc4463b8df48c8c454971/lib/libpangocairo-1.0.so"
/tmp/testdepot/artifacts/b2e490a224990d92e99cc4463b8df48c8c454971/lib/libpangoft2-1.0.so.0: undefined symbol: FcConfigSetDefaultSubstitute
Stacktrace:

and here

[ Info: PackageCompiler: Executing /tmp/test/KiteModels.jl/test/test_for_precompile.jl => /tmp/jl_packagecompiler_H4vwzh/jl_qyu1Y9
ERROR: LoadError: InitError: could not load library "/tmp/testdepot/artifacts/b2e490a224990d92e99cc4463b8df48c8c454971/lib/libpangocairo-1.0.so"
/tmp/testdepot/artifacts/b2e490a224990d92e99cc4463b8df48c8c454971/lib/libpangoft2-1.0.so.0: undefined symbol: FcConfigSetDefaultSubstitute

Summary

Errors when creating a system image due to version conflicts of shared libraries:

library error
libpangocairo-1.0.so undefined symbol: FcConfigSetDefaultSubstitute
libgio-2.0.so, libgobject-2.0.so.0 undefined symbol: g_string_copy
libpangoft2-1.0.so.0 undefined symbol: FcConfigSetDefaultSubstitute

These errors happen both locally and on CI. The workaround is to pre-load the Julia versions of these libraries. But why is this workaround needed ? Is there a bug in one of the packages?

This looks like Makie Precompilation failed in Linux and should be fixed by now.