OpenSSL_jll error after PyCall-importing a module which uses libcrypto

General Usage
, ,
1

So OpenSSL_jll uses libcrypto and libssl, both of which are downloaded as artifacts. libssl has a dependency on libcrypto, so when loading libssl the system also looks for libcrypto. During normal usage, everything works fine. However, causing (I believe) libcrypto to be loaded by a Python module with PyCall makes libssl look for libcrypto not in the artifact path, but in a system path - presumably because it’s cached from when the Python module loaded it. This causes the following:

using PyCall
pyimport("googleapiclient.http")

using OpenSSL_jll

ERROR: InitError: could not load library "/home/philip/.julia/artifacts/a696e1e3e6b8d3c2c4b67609357ebc5c35eabde1/lib/libssl.so"
/lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.3.0' not found (required by /home/philip/.julia/artifacts/a696e1e3e6b8d3c2c4b67609357ebc5c35eabde1/lib/libssl.so)
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:120
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:119
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/GfYNv/src/products/library_generators.jl:63 [inlined]
  [4] __init__()
    @ OpenSSL_jll ~/.julia/packages/OpenSSL_jll/WaTt1/src/wrappers/x86_64-linux-gnu.jl:16
  [5] run_module_init(mod::Module, i::Int64)
    @ Base ./loading.jl:1378
  [6] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1366
  [7] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any}, ignore_native::Nothing; register::Bool)
    @ Base ./loading.jl:1254
  [8] _include_from_serialized (repeats 2 times)
    @ ./loading.jl:1210 [inlined]
  [9] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128, stalecheck::Bool; reasons::Dict{String, Int64}, DEPOT_PATH::Vector{String})
    @ Base ./loading.jl:2041
 [10] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:2527
 [11] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:2388
 [12] #invoke_in_world#3
    @ ./essentials.jl:1089 [inlined]
 [13] invoke_in_world
    @ ./essentials.jl:1086 [inlined]
 [14] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:2375
 [15] macro expansion
    @ ./loading.jl:2314 [inlined]
 [16] macro expansion
    @ ./lock.jl:273 [inlined]
 [17] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:2271
 [18] #invoke_in_world#3
    @ ./essentials.jl:1089 [inlined]
 [19] invoke_in_world
    @ ./essentials.jl:1086 [inlined]
 [20] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:2260
during initialization of module OpenSSL_jll

I assume this is telling me that the system installation of libcrypto doesn’t include the version that Julia wants. Note that the code line which errors is a dlopen on libssl, after a dlopen on libcrypto has already been done. So it’s not that Julia fails to find the correct libcrypto, it’s that the system finds the wrong one when resolving the dependencies in libssl. Or at least that’s my layman’s understanding of this.

Doing the reverse, i.e.

using OpenSSL_jll

using PyCall
pyimport("googleapiclient.http")

works perfectly fine. So one workaround is to have using OpenSSL_jll in my startup.jl file, (which is what I’m currently doing), but that’s not as elegant in things like docker containers, where I instead have to make sure that the actual using statements come in a certain order. Is there a good solution to this? Can something be done in PyCall to not cache the system path, or can Julia do something to force the local libcrypto.so to be used?