PyCall imports anaconda package but fails to import package module

General Usage
1

I have built PyCall with anaconda python. I can import python packages, but cannot import submodules of the package. Here is the console output.

julia> versioninfo()
Julia Version 1.1.0
Commit 80516ca202* (2019-01-21 21:24 UTC)
Platform Info:
  OS: Linux (x86_64-redhat-linux)
  CPU: Intel(R) Core(TM) i5-6400 CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

julia> using PyCall

julia> PyCall.pyprogramname
"/home/gizem.kalender/anaconda3/bin/python"

julia> pyimport("scipy")
PyObject <module 'scipy' from '/home/gizem.kalender/anaconda3/lib/python3.7/site-packages/scipy/__init__.py'>

julia> pyimport("scipy.spatial")
ERROR: PyError (PyImport_ImportModule

The Python package scipy.spatial could not be found by pyimport. Usually this means
that you did not install scipy.spatial in the Python version being used by PyCall.

PyCall is currently configured to use the Python version at:

/home/gizem.kalender/anaconda3/bin/python

and you should use whatever mechanism you usually use (apt-get, pip, conda,
etcetera) to install the Python package containing the scipy.spatial module.

One alternative is to re-configure PyCall to use a different Python
version on your system: set ENV["PYTHON"] to the path/name of the python
executable you want to use, run Pkg.build("PyCall"), and re-launch Julia.

Another alternative is to configure PyCall to use a Julia-specific Python
distribution via the Conda.jl package (which installs a private Anaconda
Python distribution), which has the advantage that packages can be installed
and kept up-to-date via Julia.  As explained in the PyCall documentation,
set ENV["PYTHON"]="", run Pkg.build("PyCall"), and re-launch Julia. Then,
To install the scipy.spatial module, you can use `pyimport_conda("scipy.spatial", PKG)`,
where PKG is the Anaconda package the contains the module scipy.spatial,
or alternatively you can use the Conda package directly (via
`using Conda` followed by `Conda.add` etcetera).

) <class 'ImportError'>
ImportError("/usr/bin/../lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/gizem.kalender/anaconda3/lib/python3.7/site-packages/scipy/sparse/_sparsetools.cpython-37m-x86_64-linux-gnu.so)")
  File "/home/gizem.kalender/anaconda3/lib/python3.7/site-packages/scipy/spatial/__init__.py", line 98, in <module>
    from .kdtree import *
  File "/home/gizem.kalender/anaconda3/lib/python3.7/site-packages/scipy/spatial/kdtree.py", line 8, in <module>
    import scipy.sparse
  File "/home/gizem.kalender/anaconda3/lib/python3.7/site-packages/scipy/sparse/__init__.py", line 231, in <module>
    from .csr import *
  File "/home/gizem.kalender/anaconda3/lib/python3.7/site-packages/scipy/sparse/csr.py", line 15, in <module>
    from ._sparsetools import csr_tocsc, csr_tobsr, csr_count_blocks, \

Stacktrace:
 [1] pyimport(::String) at /home/gizem.kalender/.julia/packages/PyCall/ttONZ/src/PyCall.jl:544
 [2] top-level scope at none:0

julia>

Any ideas how to solve this problem?

2

If you scroll down and look at the Python ImportError exception you will see the actual error:

The problem is that Julia itself links to C++, but Anaconda’s scipy.spatial apparently wants an newer version of the libstdc++ library.

You could try using LD_PRELOAD before launching Julia to force Julia to use Anaconda’s C++ library, but in general having dependencies that link to different versions of libraries is a thorny problem.

2 Likes
3

LD_PRELOAD solved the problem. Thanks for the quick reply.