BinaryBuilder.jl can't dlopen because of libopenlibm.so

I’m running into trouble while trying to make a wrapper for the Glasgow Subgraph Solver library (C++). I’ve created a CxxWrap wrapper and verified that it works when manually compiled. Now I’m trying to get it to build with BinaryBuilder.jl. This is the error I’m getting:

$ julia build_tarballs.jl --deploy=local --verbose --debug x86_64-linux-gnu
... (lots of output) ...
[ Info: Checking shared library lib/libgssjulia.so
ERROR: Unable to load dependent library /home/dstahlke/Desktop/glasgow-julia/build/x86_64-linux-gnu/KRoj4aUJ/x86_64-linux-gnu-libgfortran5-cxx11/destdir/lib/julia/libopenlibm.so
Message:/home/dstahlke/Desktop/glasgow-julia/build/x86_64-linux-gnu/KRoj4aUJ/x86_64-linux-gnu-libgfortran5-cxx11/destdir/lib/julia/libopenlibm.so: cannot open shared object file: No such file
 or directory                                  
┌ Warning: lib/libgssjulia.so cannot be dlopen()'ed
└ @ BinaryBuilder.Auditor ~/.julia/packages/BinaryBuilder/MoPsh/src/Auditor.jl:171

If I try loading that library into julia, I get the same error. If I run my test program with LD_LIBRARY_PATH=${juliainst}/lib/julia, I get past that error but it then complains it can’t find libjulia-internal.so.1. And indeed that library doesn’t exist (${juliainst}/lib/julia only has libjulia-internal.so.1.10.0 and the symlinks libjulia-internal.so.1.10 and libjulia-internal.so).

What am I doing wrong here?

Would you be able to share the build recipe? That way it may be easier to understand what’s going on

I don’t really know what I’m doing here, so I’ve basically cobbled up some stuff using various Yggdrasil recipes as examples. Here is the BinaryBuilder.jl script:

using BinaryBuilder, Pkg

const src_name = "glasgow_subgraph_solver"
const src_version = v"0.1"

platforms = supported_platforms()
# From Yggdrasil/F/finufft/build_tarballs.jl
# Expand for microarchitectures on x86_64 (library doesn't have CPU dispatching)
#platforms = expand_cxxstring_abis(expand_microarchitectures(supported_platforms(), ["x86_64", "avx", "avx2", "avx512"]); skip=!Sys.iswindows)
#platforms = [
#    BinaryProvider.Linux(:x86_64, compiler_abi=CompilerABI(:gcc7))
#]

sources = [
    DirectorySource("/home/dstahlke/Desktop/glasgow-subgraph-solver")
]

script = raw"""
export

cmake \
      -DCMAKE_FIND_ROOT_PATH=${prefix} \
      -DCMAKE_INSTALL_PREFIX=${prefix} \
      -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN} \
      -DJulia_PREFIX=${prefix} \
      -DBUILD_SHARED_LIBS=ON \
      -DWITH_JULIA=ON \
      -S . -B build

cmake --build build --config Release --target install -- -j${nproc}

ldd ${prefix}/lib/libgssjulia.so
"""

products = [
    LibraryProduct("libgssjulia", :libgssjulia)
]

dependencies = [
    # Boost breaks ABI in every single version because they embed the full version number in
    # the SONAME, so we're compatible with one and only one version at a time.
    Dependency(PackageSpec(name="boost_jll", uuid="28df3c45-c428-5900-9ff8-a3135698ca75"); compat="=1.76.0"),
    Dependency("libcxxwrap_julia_jll"),
    BuildDependency("libjulia_jll"),
]

build_tarballs(ARGS, src_name, src_version, sources, script, platforms, products, dependencies;
    preferred_gcc_version = v"11",
    julia_compat="1.6")

In that script I’m running ldd for debug purposes. Here is the ldd output:

[15:53:07]  ---> ldd ${prefix}/lib/libgssjulia.so
[15:53:07] 	linux-vdso.so.1 (0x00007fff8f499000)
[15:53:07] 	libcxxwrap_julia.so.0 => /workspace/destdir/lib/libcxxwrap_julia.so.0 (0x00007f92a46a5000)
[15:53:07] 	libglasgow_subgraphs.so => /workspace/destdir/lib/libglasgow_subgraphs.so (0x00007f92a45b5000)
[15:53:07] 	libjulia.so.1 => /workspace/destdir/lib/libjulia.so.1 (0x00007f92a4200000)
[15:53:07] 	libboost_container.so.1.76.0 => /workspace/destdir/lib/libboost_container.so.1.76.0 (0x00007f92a3e00000)
[15:53:07] 	libstdc++.so.6 => /usr/lib/csl-glibc-x86_64/libstdc++.so.6 (0x00007f92a3bec000)
[15:53:07] 	libm.so.6 => /lib64/libm.so.6 (0x00007f92a44d9000)
[15:53:07] 	libgcc_s.so.1 => /usr/lib/csl-glibc-x86_64/libgcc_s.so.1 (0x00007f92a44be000)
[15:53:07] 	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f92a44b9000)
[15:53:07] 	libc.so.6 => /lib64/libc.so.6 (0x00007f92a39f0000)
[15:53:07] 	libdl.so.2 => /lib64/libdl.so.2 (0x00007f92a44b4000)
[15:53:07] 	librt.so.1 => /lib64/librt.so.1 (0x00007f92a44ad000)
[15:53:07] 	ldd (0x00007f92a46f7000)

So it seems at that point it’s finding everything. And it depends on libm.so rather than libopenlibm.so.

The cmake subdir for the CxxWrap wrapper that I’ve added to the C++ package:

set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})

find_package(JlCxx REQUIRED)
get_target_property(JlCxx_location JlCxx::cxxwrap_julia LOCATION)
get_filename_component(JlCxx_location ${JlCxx_location} DIRECTORY)
message(STATUS "Found JlCxx at ${JlCxx_location}")
#set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib;${JlCxx_location}")
list(APPEND CMAKE_INSTALL_RPATH "\$ORIGIN;${JlCxx_location}")

add_library(gssjulia SHARED gssjulia.cpp)
add_dependencies(gssjulia glasgow_subgraphs)
target_link_libraries(gssjulia PRIVATE JlCxx::cxxwrap_julia glasgow_subgraphs)

install(TARGETS gssjulia DESTINATION lib)

Again, I don’t really know what I’m doing here and did some copy-paste from various examples. I’m not sure whether I should be setting RPATH here. It doesn’t seem to make a difference one way or the other. It looks like either cmake or BinaryBuilder may be setting RPATH=$ORIGIN by default anyway.

When you want to link to libjulia you need to do a few things. A good simple example is the recent openfhe_julia recipe. Highlights are

openfhe_julia isn’t building for me either.

$ cd Yggdrasil/O/openfhe_julia
$ julia build_tarballs.jl --deploy=local --verbose --debug x86_64-linux-gnu
[ Info: Building and deploying version 0.2.1+1 to /home/dstahlke/git/julia_modules/dev/openfhe_julia_jll
[ Info: Building for x86_64-linux-gnu
┌ Info: Using cached git repository
│   url = "https://github.com/sloede/openfhe-julia.git"
└   repo_path = "/home/dstahlke/.julia/packages/BinaryBuilderBase/tGUXK/deps/downloads/clones/openfhe-julia.git-ff747edd651210c136334726d4b14a86a5b6a9cd31db5a6b349217e2e9be6563"
[ Info: Checking openfhe-julia.git-ff747edd651210c136334726d4b14a86a5b6a9cd31db5a6b349217e2e9be6563 out to openfhe-julia...
   Resolving package versions...
ERROR: LoadError: KeyError: key v"1.10.7" not found

If I change the libjulia_jll requested version to 1.10.8+0 (the version I have installed on my system), it gets past this point but ultimately fails with the same libopenlibm.so problem that my recipe had.

For that issue you have to use julia v1.7 (that’s what’s currently used in Yggdrasil). Note also that you may want to target the fictional platform host, which matches the current host system, instead of x86_64-linux-gnu, because host would also include the julia version.

Excellent. Running build_tarballs.jl with julia 1.7 at least gets it building.

But does this mean I can only load the modules into julia 1.7? Because that works, but I’m unable to load them into julia 1.10.

@wrapmodule(glasgow_subgraph_solver_jll.libgssjulia) fails in julia 1.10 because glasgow_subgraph_solver_jll has no member libgssjulia. And passing the path to the *.so into @wrapmodule fails with missing libjulia.so.1. Adding the missing symlink to that library, and setting LD_LIBRARY_PATH for that and for libboost, gets me to some CxxWrap error. Everything works just fine under julia 1.7 but obviously I’d prefer to be able to use the later version!

No, if you followed the instructions in my last message above and built the library for the different julia versions.

Hi, am also running into these issues. Great hints! Is there a plan to ease the situation ?
(I am well aware that this would be lots of work…)

EDIT2: Found Future Pkg compatibility · Issue #6 · JuliaPackaging/JLLPrefixes.jl · GitHub and endorse it
EDIT: Things work for me now. May be you should be aware of 0.15 not yet registered (was Regression with 0.15) · Issue #402 · JuliaInterop/CxxWrap.jl · GitHub .