Julia not finding it's own artifacts

General Usage
1

I am trying to run Circuitscape using Julia (v1.7.2) via the JuliaCall package in R and am getting the following error (as reported from within R):

Error: Error happens in Julia.
InitError: could not load library "/home/ndusek/.julia/artifacts/e4afa6442d82c9e3819067808b91aa8308d90db6/lib/libgeotiff.so"
libproj.so.25: cannot open shared object file: No such file or directory
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:117
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:117
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/QpMQW/src/products/library_generators.jl:54 [inlined]
  [4] __init__()
    @ libgeotiff_jll ~/.julia/packages/libgeotiff_jll/EG4x2/src/wrappers/x86_64-linux-gnu.jl:15
  [5] _include_from_serialized(path::String, depmods::Vector{Any})
    @ Base ./loading.jl:768
  [6] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
    @ Base ./loading.jl:854
  [7] _tryrequire_from_serialized(modkey::Base.PkgId, build_id::UInt64, modpath::String)
    @ Base ./loading.jl:783
  [8] _require_search_from_seriali

From the above, I understand that libgeotiff.so is trying to load libproj.so.25 and is not able to find it. However, I can see that Julia has an artifact for libproj.so.25 by doing a find:

$ find ~/.julia/artifacts -name libproj.so.25
/home/ndusek/.julia/artifacts/8a643038d2adde781829b4467a32afa307e23b51/lib/libproj.so.25

Why is it that Julia is unable to find it’s own copy of the library? Do I need to set some environment variable to tell Julia to use it’s own artifacts to resolve recursive library dependencies?

As an added piece of information, I am running this inside the Rocker project’s R geospatial container, so all of these geospatial libraries are available at the system level. But evidently, Julia prefers to install it’s own copy of each library.

2

I don’t know, ask your operating system which is hiding the answer from you: almost all operating systems are very bad in this regard, surprising only recent versions of macOS provide more information than absolutely nothing.

That said, do you have the LD_LIBRARY_PATH environment variable set? From within Julia you can see that with

ENV["LD_LIBRARY_PATH"]

In any case, if you want to know why the library can’t be found, you can export the environment variable LD_DEBUG=all before starting julia, and when it’ll try to dlopen libgeotiff.so you should get some messages about what is being searched. Note: LD_DEBUG=all will print a lot of text to standard error, you may want to redirect it to file to analyze it later.

3

Hi @giordano, thanks for the quick reply and for being patient with me.

What you’ve described is what I would normally do, i.e. add the lib dir to LD_LIBRARY_PATH. In this case, I assumed since Julia was installing libraries in ~/.julia/artifacts that adding $JULIA_HOME/lib to the LD_LIBRARY_PATH wouldn’t do the trick. But in fact, it did (mea culpa).

So there must be some magic Julia is doing behind the scenes whereby adding Julia’s global lib dir to LD_LIBRARY_PATH in turn links all the libs in ~/.julia/artifacts?

In any case, the original error is resolved after making that chance. Now I’m getting errors from GDAL, but I think these might be related to the interop with R and not Julia per se.

Thanks for the help.

4

There isn’t much magic involved, but if your LD_LIBRARY_PATH wasn’t empty (or with some non-julia libraries taking precedence over julia directories), julia may have mixed up some libraries causing troubles down the line. I generally recommend starting Julia with an empty LD_LIBRARY_PATH if possible, in that way you avoid mixing up julia’s libraries with system libraries.

1 Like