Could not load library librsvg - very strange error

#1

I’m using VegaLite library, and after some update it does not load when imported after PyPlot:

using PyPlot
using VegaLite

gives the following:

ERROR: InitError: error compiling __init__: could not load library "/usr/lib/x86_64-linux-gnu/librsvg-2.so.2"
/lib/x86_64-linux-gnu/libgio-2.0.so.0: undefined symbol: g_spawn_async_with_fds
Stacktrace:
 [1] _include_from_serialized(::String, ::Array{Any,1}) at ./loading.jl:633
 [2] _require_search_from_serialized(::Base.PkgId, ::String) at ./loading.jl:713
 [3] _tryrequire_from_serialized(::Base.PkgId, ::UInt64, ::String) at ./loading.jl:648
 [4] _require_search_from_serialized(::Base.PkgId, ::String) at ./loading.jl:702
 [5] _require(::Base.PkgId) at ./loading.jl:937
 [6] require(::Base.PkgId) at ./loading.jl:858
 [7] require(::Module, ::Symbol) at ./loading.jl:853
during initialization of module Rsvg

Following to Rsvg.jl __init__ method I narrowed this error down to this code:

julia -E 'using PyPlot; ccall((:rsvg_set_default_dpi, "librsvg-2.so.2"), Nothing, (Float64,), 1.)'

gives the same error, and executes fine if I remove using PyPlot. The issue seems related to some interaction with python and plots, as the following line (for example) also gives the same error:

julia -E 'using PyCall; pyimport("seaborn"); ccall((:rsvg_set_default_dpi, "librsvg-2.so.2"), Nothing, (Float64,), 1.)'

At the same time,

julia -E 'ccall((:rsvg_set_default_dpi, "librsvg-2.so.2"), Nothing, (Float64,), 1.)'

correctly prints nothing without errors.

Any idea how to debug and fix this?

#2

I’m not an expert on this, but here is my suspicion: PyPlot.jl somehow loads a system librsvg, and then when Rsvg.jl later tries to load its own copy of librsvg things go wrong: because a librsvg is already loaded, Rsvg.jl won’t get the version of librsvg that it downloaded via its build.jl script, but instead it gets the system version of the library that was loaded by PyPlot.jl, and that is somehow a different version that is incompatible. Or I might be on a completely wrong track here :slight_smile:

I’m going to ping some folks that either might be able to understand this better than I do, or are associated with packages that play a role here: @lobingera, @stevengj, @staticfloat.

#3

Oh, and just for good measure I’ll repeat my general plea: it would be fantastic if someone would port the Gtk.jl/Cairo.jl/Rsvg.jl stack over to BinaryBuilder.jl. I don’t even know whether that would help here, but hey, I might as well mention this again :wink:

#4

I think David is partially right, a certain .so has been loaded and in the process of opening librsvg this has a mismatch. My candiate of choice would be libgio which has been recently added to Rsvg.jl for handling of huge svgs.

#5

@staticfloat is there a plan for situations like this? What can we do to fix this?

#6

In the short term, try explicitly dlopen()'ing the libgio you want to use before running using PyPlot.

#7

This workaround isn’t even needed in my case - I just do using VegaLite first, and using PyPlot after that. The question is about this situation in general, because it looks quite strange and difficult to debug - think if there were more than two libraries involved.

#8

@staticfloat, just out of curiosity, is there a long term plan to address this?

#9

Yes, the long-term plan is https://github.com/JuliaLang/Pkg.jl/issues/841

#10

@staticfloat how would the problem discussed here in this thread be addressed by https://github.com/JuliaLang/Pkg.jl/issues/841? The problem here in this thread seems to be that something that is not under BinaryProvider control loads a version of shared library libA, and then a package that uses BinaryProvider.jl that also needs libA never gets the version that it requires, but instead the version that the non-BinaryProvider package loaded. Would that specifically be resolved with that new proposal? I didn’t really see anything in that thread that addressed this specific situation.

#11

Does BB support cargo + rust?

#12

I didn’t really see anything in that thread that addressed this specific situation.

Sorry, I think I got two wires crossed in my head; you’re right, that is tangentially related at best (it is related to ensuring that dependent packages can load their dependencies properly) but doesn’t address the core issue here.

This is not a julia-specific problem, and is very complex due to the interaction of Python, whose behavior we cannot regulate as closely as that of Julia packages. The fundamental issue is that we are loading two different library versions, but our typical efforts to control which versions get loaded (the link I placed above) won’t work here, because Python can do whatever it wants; it knows nothing about Pkg or BinaryProvider, etc…

The OS, in general, will try to steer you clear of loading multiple copies of the same dynamic library into your address space (e.g. if you try to load something called libgio and it’s already loaded, it will just return immediately; but if you say "load /path/to/libgio.2.3.1.so", it will actually try to load that library) so depending on how python and julia try to load these libraries (and which versions of the libraries are available on your system) you can get very different behavior by re-ordering your statements.

The best advice I can give is to try and load everything from Julia side first; beyond that I’m not sure what we can do, because importing python into Julia’s address space and allowing it to load libraries means that it can really do whatever it wants, and we can’t really control it.