`Libdl.dlopen` versus `dlfcn.h`'s `dlopen`

Internals & Design Internals
1

On my system, I see

// Build with
// `gcc test_libpython.c -ldl`

#include <stdio.h>
#include <dlfcn.h>

int main() {
    void* handle = dlopen("/usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0", RTLD_LAZY | RTLD_DEEPBIND | RTLD_GLOBAL);
    if (!handle) {
        fprintf(stderr, "%s\n", dlerror());
        return 1;
    }
    printf("Library loaded successfully\n");
    dlclose(handle);
    return 0;
}

$ gcc test_libpython.c -ldl
$ ./a.out 
Library loaded successfully

but I see

julia> using Libdl: dlpath, dlopen, dlopen_e, dlclose, dlsym, dlsym_e, RTLD_LAZY, RTLD_DEEPBIND, RTLD_GLOBAL

julia> lib_path = "/usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0"
"/usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0"

julia> dlopen_flags = RTLD_LAZY | RTLD_DEEPBIND | RTLD_GLOBAL
0x00000046

julia> dlopen(lib_path, dlopen_flags)
ERROR: could not load library "/usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0"
libexpat.so.1: cannot open shared object file: No such file or directory
Stacktrace:
 [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
   @ Base.Libc.Libdl ./libdl.jl:117
 [2] dlopen(s::String, flags::UInt32)
   @ Base.Libc.Libdl ./libdl.jl:116
 [3] top-level scope
   @ REPL[4]:1

shell> ldd /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0
	linux-vdso.so.1 (0x00007ffc33319000)
	libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007fa6b229e000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fa6b2282000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa6b219b000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa6b1f70000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fa6b28cd000)

julia> dlopen("/lib/x86_64-linux-gnu/libexpat.so.1", dlopen_flags)
Ptr{Nothing} @0x0000000007dec090

julia> dlopen(lib_path, dlopen_flags)
Ptr{Nothing} @0x0000000007c7d0c0

Note that libpython does load correctly after loading libexpat explicitly.

This is within an Ubuntu docker image, so I imagine something is misconfigured, but it is hard to understand what since the “analogous” C code is working. Or so I think. There is much more going on in jl_load_dynamic_library, so I don’t know what the right test is.

2

Does it work if you add /lib/x86_64-linux-gnu to DL_LOAD_PATH?

3

jl_dlopen calls the real dlopen here:

Have you checked if the LD_LIBRARY_PATH and PATH environment variables are the same in both circumstances?

You may want to run julia via LD_DEBUG=all julia to diagnose.

Chdck the manpage for additional options:
https://man7.org/linux/man-pages/man8/ld.so.8.html

4

This was helpful. It reminded me that Julia was installed via Nix in this environment, with output like


    197293:	symbol=stderr;  lookup in file=/nix/store/8mc30d49ghc8m5z96yz39srlhg5s9sjj-glibc-2.38-44/lib/libc.so.6 [0]

and that is why it couldn’t find any of the system libraries that were not explicit dependencies.

Thanks!

1 Like