# Precedence of local and julia-shipped shared libraries

**URL:** <https://discourse.julialang.org/t/precedence-of-local-and-julia-shipped-shared-libraries/104258>\
**Category:** General Usage\
**Tags:** package-compiler\
**Created:** [September 26, 2023, 1:09pm UTC](https://discourse.julialang.org/t/precedence-of-local-and-julia-shipped-shared-libraries/104258 "2023-09-26T13:09:36Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Benedict](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/benedict/32/208694_2.png) [@Benedict](https://discourse.julialang.org/u/Benedict)\
**Post date:** [September 26, 2023, 1:09pm UTC](https://discourse.julialang.org/t/precedence-of-local-and-julia-shipped-shared-libraries/104258/1 "2023-09-26T13:09:36Z")

</div>

I am working together with @sloede on [libtrixi](https://github.com/trixi-framework/libtrixi), a C/Fortran-interface library to the flow solver `Trixi.jl`. @sloede managed to create a standalone library using `PackageCompiler.jl`, see [How do libraries compiled by PackageCompiler.jl interact with Preferences.jl?](https://discourse.julialang.org/t/how-do-libraries-compiled-by-packagecompiler-jl-interact-with-preferences-jl/103479)  
When using it, I encountered a problem with different versions of local and julia-shipped libraries. In detail:

- Our Julia project uses `Preferences.jl` to create `LocalPreferences.toml` with

- This library was compiled locally and links against `libstdc++`:

- `PackageCompiler.jl` fetches `libstdc++.so.6.0.30`

- When executing an example using the `PackageCompiler.jl` standalone library, we get an error

- We can work around this by using `LD_PRELOAD=/usr/lib/libstdc++.so.6.0.32`

Now we wonder whether it be possible to detect and avoid such conflicts. Would it possible to override the julia-shipped libraries during PackageCompiler.jl runs?

cc @giordano @sloede @vchuravy

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [September 26, 2023, 1:21pm UTC](https://discourse.julialang.org/t/precedence-of-local-and-julia-shipped-shared-libraries/104258/2 "2023-09-26T13:21:46Z")

</div>

What version of Julia are you using?

---

<div class="post-metadata">

**Author:** ![Benedict](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/benedict/32/208694_2.png) [@Benedict](https://discourse.julialang.org/u/Benedict)\
**Post date:** [September 26, 2023, 1:30pm UTC](https://discourse.julialang.org/t/precedence-of-local-and-julia-shipped-shared-libraries/104258/3 "2023-09-26T13:30:23Z")

</div>

Sorry, I forgot to mention that. julia 1.9.3

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [September 26, 2023, 1:57pm UTC](https://discourse.julialang.org/t/precedence-of-local-and-julia-shipped-shared-libraries/104258/4 "2023-09-26T13:57:45Z")

</div>

Julia v1.9.3 is able to pick up libstdc++ newer than the one shipped by Julia:

```julia
% julia -q
julia> using Libdl

julia> filter!(contains("libstdc++"), dllist())
1-element Vector{String}:
 "/usr/lib/libstdc++.so.6"

```

I have no idea what PackageCompiler does.

---

<div class="post-metadata">

**Author:** ![sloede](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sloede/32/44787_2.png) [@sloede](https://discourse.julialang.org/u/sloede)\
**Post date:** [September 26, 2023, 2:21pm UTC](https://discourse.julialang.org/t/precedence-of-local-and-julia-shipped-shared-libraries/104258/5 "2023-09-26T14:21:59Z")

</div>

I am not even sure if this is related to PackageCompiler.jl or just becomes visible here because the load order is different here than it would be for a run where `julia` is the main executable (although I don’t see how).

It seems like `libstdc++` is required by `libjulia-internal.so.1` only, and that has `$ORIGIN:$ORIGIN/..` set as the `RUNPATH`. So one - crude - solution could be to

- create a new folder `PREFIX/lib/julia/override`
- use `patchelf` to change the `RUNPATH` for `libjulia-internal.so.1` to `$ORIGIN/../julia/override:$ORIGIN:$ORIGIN/..`,
- and then symlink `/usr/lib/libstdc++.so.6` to `PREFIX/lib/julia/override/libstdc++.so.6`.

But it feels like there should exist a better way to handle that.

For user convenience, we are looking for a solution that either prevents this problem in the first place by fixing whatever is wrong in PackageCompiler.jl/our own compilation part, or that can be (semi-)auto-detected and fixed in a post processing step. We do not want the user to have to recompile with PackageCompiler.jl (too long) nor force them to always start their problems with `LD_PRELOAD=...` (too error prone).

---

<div class="post-metadata">

**Author:** ![staticfloat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/staticfloat/32/34_2.png) [@staticfloat](https://discourse.julialang.org/u/staticfloat)\
**Post date:** [September 26, 2023, 4:09pm UTC](https://discourse.julialang.org/t/precedence-of-local-and-julia-shipped-shared-libraries/104258/6 "2023-09-26T16:09:08Z")

</div>

We have [some magic in the Julia loader](https://github.com/JuliaLang/julia/blob/8de80bdf902e0899dd3e39636a95769cf44c823b/cli/loader_lib.c#L185-L353) that basically [tries to load the first `libstdc++.so` that it can find](https://github.com/JuliaLang/julia/blob/8de80bdf902e0899dd3e39636a95769cf44c823b/cli/loader_lib.c#L271-L279) (and this is being run from the `julia` executable, which does _not_ have `RUNPATH` set, so it picks up the system-wide `libstdc++` if it exists), checks to see if that version is new enough to run Julia, and if so, uses that. If that system-wide `libstdc++` does not exist or is too old, we will [instead load the bundled `libstdc++`](https://github.com/JuliaLang/julia/blob/8de80bdf902e0899dd3e39636a95769cf44c823b/cli/loader_lib.c#L464-L468).

The “correct” way to do this is probably to copy this `libstdcxxprobe()` functionality into `PackageCompiler.jl`.

---

<div class="post-metadata">

**Author:** ![sloede](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sloede/32/44787_2.png) [@sloede](https://discourse.julialang.org/u/sloede)\
**Post date:** [September 26, 2023, 8:31pm UTC](https://discourse.julialang.org/t/precedence-of-local-and-julia-shipped-shared-libraries/104258/7 "2023-09-26T20:31:49Z")

</div>

Interesting 🤔 So you’re saying that Julia will always dynamically determine at startup which is the “best” `libstdc++.so` so load?

While this makes sense when using Julia as the main program, I think for a compiled library it would be sufficient if the correct `libstdc++.so` would be determined (and fixed) at build time. To this effect, I don’t think we would have to copy all the C code over to PackageCompiler.jl, but rather just figure out WWJD (what would Julia do) by running something like the following from PackageCompiler.jl:

```julia
using Libdl
libstdcpp_path = filter!(contains("libstdc++"), dllist()) |> first |> abspath

```

One could then determine whether `libstdcpp_path` points to the private libjulia dir (`julia_private_libdir()` in PC.jl) and proceed based on the outcome:

- if yes (e.g., ` /opt/julia/1.9.3/lib/julia/libstdc++.so.6`), copy over libstdc++ from Julia as usual
- if no (e.g., `/usr/lib/x86_64-linux-gnu/libstdc++.so.6`), create a symlink to it in the private libjulia dir

Would this be a sensible option/default for PackageCompiler.jl?

---

<div class="post-metadata">

**Author:** ![staticfloat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/staticfloat/32/34_2.png) [@staticfloat](https://discourse.julialang.org/u/staticfloat)\
**Post date:** [September 26, 2023, 9:49pm UTC](https://discourse.julialang.org/t/precedence-of-local-and-julia-shipped-shared-libraries/104258/8 "2023-09-26T21:49:29Z")

</div>

I think it could make sense to “bake in” the choice of `libstdc++` that Julia has made; although I don’t think symlinks are a good idea, since presumably if someone is using `PackageCompiler.jl` they will want to redistribute the package.

---

<div class="post-metadata">

**Author:** ![sloede](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sloede/32/44787_2.png) [@sloede](https://discourse.julialang.org/u/sloede)\
**Post date:** [September 26, 2023, 10:12pm UTC](https://discourse.julialang.org/t/precedence-of-local-and-julia-shipped-shared-libraries/104258/9 "2023-09-26T22:12:27Z")

</div>

> [@staticfloat](#):
>
> I don’t think symlinks are a good idea, since presumably if someone is using `PackageCompiler.jl` they will want to redistribute the package.

I think we are leaving the realm of true portability across systems here anyways. OTOH, `libstdc++.so` is \<5 MiB, so this “macht den Kohl auch nicht mehr fett” (this doesn’t fatten the cabbage, as the Germans say; i.e., by now it does not matter anymore anyways [looking suspiously at you, 500 MiB `libtrixi.so`]).

> [@staticfloat](#):
>
> I think it could make sense to “bake in” the choice of `libstdc++` that Julia has made

Thanks for your feedback. I’ll give it a try to see how ugly this would get in PC.jl.

---

<div class="post-metadata">

**Author:** ![staticfloat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/staticfloat/32/34_2.png) [@staticfloat](https://discourse.julialang.org/u/staticfloat)\
**Post date:** [September 26, 2023, 10:21pm UTC](https://discourse.julialang.org/t/precedence-of-local-and-julia-shipped-shared-libraries/104258/10 "2023-09-26T22:21:51Z")

</div>

> [@sloede](#):
>
> I think we are leaving the realm of true portability across systems here anyways.

Really? I think that is one of the primary use cases for PackageCompiler.jl! I’m curious what you’re using this for if you’re not planning to ship the built library?

---

<div class="post-metadata">

**Author:** ![sloede](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sloede/32/44787_2.png) [@sloede](https://discourse.julialang.org/u/sloede)\
**Post date:** [September 26, 2023, 10:46pm UTC](https://discourse.julialang.org/t/precedence-of-local-and-julia-shipped-shared-libraries/104258/11 "2023-09-26T22:46:49Z")

</div>

> [@staticfloat](#):
>
> > [@sloede](#):
> >
> > I think we are leaving the realm of true portability across systems here anyways.
> 
> Really?

Maybe I should qualify my comment first: bundling a system-local libstdc++ from one Linux system and hoping it will just work anywhere else, is IMHO quite daring (but I’m no expert, maybe this is actually a sane thing to do/expect).

> [@staticfloat](#):
>
> I’m curious what you’re using this for if you’re not planning to ship the built library?

We are developing the [libtrixi](https://github.com/trixi-framework/libtrixi) library as a means to allowing one to control numerical simulations with [Trixi.jl](https://github.com/trixi-framework/Trixi.jl) from C/C++/Fortran programs. Since libtrixi relies on system-local installations of third-party libraries such as MPI, HDF5, and others, and since the paths to these libraries are usually baked into the compiled library as compile-time preferences, I have zero expectation of system portability for our use case.

But I think that’s OK; in our community nobody expects binaries to be portable.

Incidentally, do you know why Julia’s libstdc++ is ~18 MB and Ubuntu’s is just 2 MB?

---

<div class="post-metadata">

**Author:** ![staticfloat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/staticfloat/32/34_2.png) [@staticfloat](https://discourse.julialang.org/u/staticfloat)\
**Post date:** [September 26, 2023, 11:41pm UTC](https://discourse.julialang.org/t/precedence-of-local-and-julia-shipped-shared-libraries/104258/12 "2023-09-26T23:41:05Z")

</div>

> Incidentally, do you know why Julia’s libstdc++ is ~18 MB and Ubuntu’s is just 2 MB?

If you `strip` it, it gets much closer (2.5 MB). We usually do that automatically for BB-built binaries, but the compiler libraries are special, so they don’t get the same treatment; that would be a good improvement.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [September 27, 2023, 12:25am UTC](https://discourse.julialang.org/t/precedence-of-local-and-julia-shipped-shared-libraries/104258/13 "2023-09-27T00:25:39Z")

</div>

> [@staticfloat](#):
>
> We usually do that automatically for BB-built binaries

Do we? 🤔 If we did, libstdc++ would be stripped in CompilerSupportLibraries\_jll.

---

<div class="post-metadata">

**Author:** ![sloede](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sloede/32/44787_2.png) [@sloede](https://discourse.julialang.org/u/sloede)\
**Post date:** [September 27, 2023, 10:40am UTC](https://discourse.julialang.org/t/precedence-of-local-and-julia-shipped-shared-libraries/104258/14 "2023-09-27T10:40:44Z")

</div>

I’ve created a PR to PackageCompiler.jl: [Bundle dynamically selected libstdc++ by sloede · Pull Request #853 · JuliaLang/PackageCompiler.jl · GitHub](https://github.com/JuliaLang/PackageCompiler.jl/pull/853)