Precedence of local and julia-shipped shared libraries

I am working together with @sloede on libtrixi, a C/Fortran-interface library to the flow solver Trixi.jl. @sloede managed to create a standalone library using PackageCompiler.jl, see How do libraries compiled by PackageCompiler.jl interact with Preferences.jl?
When using it, I encountered a problem with different versions of local and julia-shipped libraries. In detail:

  • Our Julia project uses Preferences.jl to create LocalPreferences.toml with

    libt8 = "/home/install/t8code/lib/"
  • This library was compiled locally and links against libstdc++:

    $ ldd /home/install/t8code/lib/
 => /usr/lib/

    On my machine this is

  • PackageCompiler.jl fetches

  • When executing an example using the PackageCompiler.jl standalone library, we get an error

    InitError(mod=:Trixi, error=ErrorException("could not load library "/home/install/t8code/lib/" 
    /home/libtrixi/LibTrixi.jl/lib/build/lib/julia/ version `GLIBCXX_3.4.32' not found   (required by /home/install/t8code/lib/"))

    because GLIBCXX_3.4.32 is only found in the system, but not in the julia-shipped

  • We can work around this by using LD_PRELOAD=/usr/lib/

Now we wonder whether it be possible to detect and avoid such conflicts. Would it possible to override the julia-shipped libraries during PackageCompiler.jl runs?

cc @giordano @sloede @vchuravy

What version of Julia are you using?

Sorry, I forgot to mention that. julia 1.9.3

Julia v1.9.3 is able to pick up libstdc++ newer than the one shipped by Julia:

% julia -q
julia> using Libdl

julia> filter!(contains("libstdc++"), dllist())
1-element Vector{String}:

I have no idea what PackageCompiler does.

1 Like

I am not even sure if this is related to PackageCompiler.jl or just becomes visible here because the load order is different here than it would be for a run where julia is the main executable (although I don’t see how).

It seems like libstdc++ is required by only, and that has $ORIGIN:$ORIGIN/.. set as the RUNPATH. So one - crude - solution could be to

  • create a new folder PREFIX/lib/julia/override
  • use patchelf to change the RUNPATH for to $ORIGIN/../julia/override:$ORIGIN:$ORIGIN/..,
  • and then symlink /usr/lib/ to PREFIX/lib/julia/override/

But it feels like there should exist a better way to handle that.

For user convenience, we are looking for a solution that either prevents this problem in the first place by fixing whatever is wrong in PackageCompiler.jl/our own compilation part, or that can be (semi-)auto-detected and fixed in a post processing step. We do not want the user to have to recompile with PackageCompiler.jl (too long) nor force them to always start their problems with LD_PRELOAD=... (too error prone).

We have some magic in the Julia loader that basically tries to load the first that it can find (and this is being run from the julia executable, which does not have RUNPATH set, so it picks up the system-wide libstdc++ if it exists), checks to see if that version is new enough to run Julia, and if so, uses that. If that system-wide libstdc++ does not exist or is too old, we will instead load the bundled libstdc++.

The “correct” way to do this is probably to copy this libstdcxxprobe() functionality into PackageCompiler.jl.

Interesting :thinking: So you’re saying that Julia will always dynamically determine at startup which is the “best” so load?

While this makes sense when using Julia as the main program, I think for a compiled library it would be sufficient if the correct would be determined (and fixed) at build time. To this effect, I don’t think we would have to copy all the C code over to PackageCompiler.jl, but rather just figure out WWJD (what would Julia do) by running something like the following from PackageCompiler.jl:

using Libdl
libstdcpp_path = filter!(contains("libstdc++"), dllist()) |> first |> abspath

One could then determine whether libstdcpp_path points to the private libjulia dir (julia_private_libdir() in PC.jl) and proceed based on the outcome:

  • if yes (e.g., /opt/julia/1.9.3/lib/julia/, copy over libstdc++ from Julia as usual
  • if no (e.g., /usr/lib/x86_64-linux-gnu/, create a symlink to it in the private libjulia dir

Would this be a sensible option/default for PackageCompiler.jl?

I think it could make sense to “bake in” the choice of libstdc++ that Julia has made; although I don’t think symlinks are a good idea, since presumably if someone is using PackageCompiler.jl they will want to redistribute the package.

I think we are leaving the realm of true portability across systems here anyways. OTOH, is <5 MiB, so this “macht den Kohl auch nicht mehr fett” (this doesn’t fatten the cabbage, as the Germans say; i.e., by now it does not matter anymore anyways [looking suspiously at you, 500 MiB]).

Thanks for your feedback. I’ll give it a try to see how ugly this would get in PC.jl.

Really? I think that is one of the primary use cases for PackageCompiler.jl! I’m curious what you’re using this for if you’re not planning to ship the built library?

Maybe I should qualify my comment first: bundling a system-local libstdc++ from one Linux system and hoping it will just work anywhere else, is IMHO quite daring (but I’m no expert, maybe this is actually a sane thing to do/expect).

We are developing the libtrixi library as a means to allowing one to control numerical simulations with Trixi.jl from C/C++/Fortran programs. Since libtrixi relies on system-local installations of third-party libraries such as MPI, HDF5, and others, and since the paths to these libraries are usually baked into the compiled library as compile-time preferences, I have zero expectation of system portability for our use case.

But I think that’s OK; in our community nobody expects binaries to be portable.

Incidentally, do you know why Julia’s libstdc++ is ~18 MB and Ubuntu’s is just 2 MB?

Incidentally, do you know why Julia’s libstdc++ is ~18 MB and Ubuntu’s is just 2 MB?

If you strip it, it gets much closer (2.5 MB). We usually do that automatically for BB-built binaries, but the compiler libraries are special, so they don’t get the same treatment; that would be a good improvement.

Do we? :thinking: If we did, libstdc++ would be stripped in CompilerSupportLibraries_jll.

I’ve created a PR to PackageCompiler.jl: Bundle dynamically selected libstdc++ by sloede · Pull Request #853 · JuliaLang/PackageCompiler.jl · GitHub

1 Like