SimpleITK via PythonCall not loading NumPy

General Usage
,
1

I am trying to use SimpleITK via PythonCall inside a Pluto notebook. I want to load a DICOM image and then convert it to a numpy array. The code to do this is straightforward and works in pure python, but with PythonCall this produces an error.

using CondaPkg, DICOM, PythonCall
CondaPkg.add("SimpleITK")
CondaPkg.add("numpy")
sitk = pyimport("SimpleITK")
np = pyimport("numpy")


root_path = "/Users/daleblack/Library/CloudStorage/GoogleDrive-djblack@uci.edu/My Drive/Datasets/perfusion/limb"
dicom_path = joinpath(root_path, "DICOM")
fixed_dicom_dir = joinpath(dicom_path, "01")


function load_dicom_sitk(path)
	series_ids = sitk.ImageSeriesReader.GetGDCMSeriesIDs(path)
	series_file_names = sitk.ImageSeriesReader.GetGDCMSeriesFileNames(path, series_ids[0])
	
	reader = sitk.ImageSeriesReader()
	reader.SetFileNames(series_file_names)
	image = reader.Execute()
	image = sitk.Cast(image, sitk.sitkFloat32)
end

fixed_image = load_dicom_sitk(fixed_dicom_dir)
sitk.GetArrayFromImage(fixed_image)

With this error

Python: ImportError: NumPy not available.

pythrow()@err.jl:94
PythonCall@err.jl:10[inlined]
PythonCall@object.jl:210[inlined]
var"#pycall#59"(::Base.Pairs{Symbol, Union{}, Tuple{}, @NamedTuple{}}, ::typeof(PythonCall.pycall), ::PythonCall.Py, ::PythonCall.Py)@object.jl:228
Main@object.jl:218[inlined]
Main@Py.jl:341[inlined]
Main@Local: 1[inlined]

Any idea how to get this to work? I want to be able to convert a Julia array to an sitk object and to do this I need to go through numpy I think

2

It also errors in the other direction, trying to convert julia array to sitk Image

function load_dcm_array(dcm_data::Vector{DICOM.DICOMData})
    return array = cat(
        [dcm_data[i][tag"Pixel Data"] for i in 1:length(dcm_data)]...; dims=3
    )
end

dcms_jl = DICOM.dcmdir_parse(fixed_dicom_dir);
fixed_image_jl = load_dcm_array(dcms_jl);
fixed_image_pyarr = PyArray(fixed_image_jl);

sitk.GetImageFromArray(fixed_image_pyarr)


Python: ImportError: Numpy not available.

pythrow()@err.jl:94
PythonCall@err.jl:10[inlined]
PythonCall@object.jl:210[inlined]
var"#pycall#59"(::Base.Pairs{Symbol, Union{}, Tuple{}, @NamedTuple{}}, ::typeof(PythonCall.pycall), ::PythonCall.Py, ::PythonCall.PyArray{Int16, 3, true, true, Int16})@object.jl:228
Main@object.jl:218[inlined]
Main@Py.jl:341[inlined]
Main@Local: 1[inlined]
3

I figured out that I get the same error if I run this in a Jupyter notebook using Python kernel. And after restarting the kernel, the error goes away. Maybe something like that is occurring in Julia?

4

When you import PythonCall.jl at the start, an environment with Python 3.11 is created:

  Package               Version  Build               Channel           Size
─────────────────────────────────────────────────────────────────────────────
  Install:
─────────────────────────────────────────────────────────────────────────────

  + libstdcxx-ng         11.4.0  h4dcbe23_1          conda-forge     Cached
  + _libgcc_mutex           0.1  conda_forge         conda-forge     Cached
  + ld_impl_linux-64       2.40  h41732ed_0          conda-forge     Cached
  + ca-certificates   2023.7.22  hbcca054_0          conda-forge     Cached
  + libgomp              13.2.0  h807b86a_1          conda-forge     Cached
  + _openmp_mutex           4.5  2_gnu               conda-forge     Cached
  + libgcc-ng            13.2.0  h807b86a_1          conda-forge     Cached
  + openssl               3.1.3  hd590300_0          conda-forge     Cached
  + libzlib              1.2.13  hd590300_5          conda-forge     Cached
  + libffi                3.4.2  h7f98852_5          conda-forge     Cached
  + bzip2                 1.0.8  h7f98852_4          conda-forge     Cached
  + ncurses                 6.4  hcb278e6_0          conda-forge     Cached
  + libuuid              2.38.1  h0b41bf4_0          conda-forge     Cached
  + libexpat              2.5.0  hcb278e6_1          conda-forge     Cached
  + xz                    5.2.6  h166bdaf_0          conda-forge     Cached
  + libnsl                2.0.0  h7f98852_0          conda-forge     Cached
  + libsqlite            3.43.0  h2797004_0          conda-forge     Cached
  + tk                   8.6.12  h27826a3_0          conda-forge     Cached
  + readline                8.2  h8228510_1          conda-forge     Cached
  + tzdata                2023c  h71feb2d_0          conda-forge     Cached
  + python               3.11.5  hab00c5b_0_cpython  conda-forge     Cached
  + wheel                0.41.2  pyhd8ed1ab_0        conda-forge     Cached
  + setuptools           68.2.2  pyhd8ed1ab_0        conda-forge     Cached
  + pip                  23.2.1  pyhd8ed1ab_0        conda-forge     Cached

Then, when SimpleITK is added to the conda environment, a different Python version is installed

  Package           Version  Build                Channel           Size
──────────────────────────────────────────────────────────────────────────
  Install:
──────────────────────────────────────────────────────────────────────────

  + expat             2.5.0  hcb278e6_1           conda-forge     Cached
  + zlib             1.2.13  hd590300_5           conda-forge     Cached
  + python_abi          3.9  4_cp39               conda-forge     Cached
  + jpeg                 9e  h0b41bf4_3           conda-forge     Cached
  + libpng           1.6.39  h753d276_0           conda-forge     Cached
  + libdeflate         1.10  h7f98852_0           conda-forge     Cached
  + libgfortran4      7.5.0  h14aa051_20          conda-forge     Cached
  + jbig                2.1  h7f98852_2003        conda-forge     Cached
  + libwebp-base      1.3.2  hd590300_0           conda-forge     Cached
  + lz4-c             1.9.3  h9c3ff4c_1           conda-forge     Cached
  + lerc                3.0  h9c3ff4c_0           conda-forge     Cached
  + eigen             3.4.0  h4bd325d_0           conda-forge     Cached
  + libgfortran-ng    7.5.0  h14aa051_20          conda-forge     Cached
  + zstd              1.5.2  h8a70e8d_1           conda-forge     Cached
  + fftw              3.3.8  nompi_hfc0cae8_1114  conda-forge     Cached
  + hdf5             1.10.6  nompi_h3c11f04_101   conda-forge     Cached
  + libtiff           4.3.0  h542a066_3           conda-forge     Cached
  + libitk            5.1.2  h86d312b_0           conda-forge     Cached
  + libitk-devel      5.1.2  h9c3ff4c_0           conda-forge     Cached
  + simpleitk         2.0.2  py39h156e488_1       conda-forge     Cached

  Reinstall:
──────────────────────────────────────────────────────────────────────────

  o libstdcxx-ng     11.4.0  h4dcbe23_1           conda-forge     Cached
  o wheel            0.41.2  pyhd8ed1ab_0         conda-forge     Cached
  o setuptools       68.2.2  pyhd8ed1ab_0         conda-forge     Cached
  o pip              23.2.1  pyhd8ed1ab_0         conda-forge     Cached

  Downgrade:
──────────────────────────────────────────────────────────────────────────

  - python           3.11.5  hab00c5b_0_cpython   conda-forge     Cached
  + python           3.9.18  h0755675_0_cpython   conda-forge     Cached

I have only briefly looked at how PythonCall.jl links to Python, but I think that since it is already imported, PythonCall.jl is pointing to the original Python version (3.11).

julia> PythonCall.C.CTX.lib_path
"/tmp/jl_0SO6uT/.CondaPkg/env/lib/libpython3.11.so.1.0"

One fix would be to create the environment before importing PythonCall.jl. This can be done with a CondaPkg.toml file, or the following modification to your code:

using CondaPkg
CondaPkg.add("SimpleITK", resolve=false)
CondaPkg.add("numpy")

using DICOM, PythonCall
sitk = pyimport("SimpleITK")
np = pyimport("numpy")


root_path = "/Users/daleblack/Library/CloudStorage/GoogleDrive-djblack@uci.edu/My Drive/Datasets/perfusion/limb"
dicom_path = joinpath(root_path, "DICOM")
fixed_dicom_dir = joinpath(dicom_path, "01")


function load_dicom_sitk(path)
	series_ids = sitk.ImageSeriesReader.GetGDCMSeriesIDs(path)
	series_file_names = sitk.ImageSeriesReader.GetGDCMSeriesFileNames(path, series_ids[0])
	
	reader = sitk.ImageSeriesReader()
	reader.SetFileNames(series_file_names)
	image = reader.Execute()
	image = sitk.Cast(image, sitk.sitkFloat32)
end

fixed_image = load_dicom_sitk(fixed_dicom_dir)
sitk.GetArrayFromImage(fixed_image)
3 Likes
5

@tylerjthomas9 is exactly right.

This is a known (but undocumented) gotcha - similar to how you shouldn’t install Julia packages after importing anything (it can lead to incompatible packages) - only in this case it’s the python interpreter itself which is incompatible, with more disastrous consequences.

Suggestions of how to help avoid this welcome.

3 Likes
6

I.e. 3.9, but it’s unclear to me why. SimpleITK supports 3.11, and in Conda, i.e. conda-forge, 3.11 is support (as opposed to Anaconda, which supports at least 3.10 seemingly).

Python 3.9 should still work (with or without Julia).

7

What version of Julia are you using? PythonCall.jl adds libstdcxx-ng bounds in conda to ensure that the Python packages are compatible with Julia. This means that the latest Python package versions may not be installed when using older Julia versions since there would be incompatibilities. With Julia v1.9.3, your original code runs for me without any errors.

Additionally, CondaPkg.jl uses the channel conda-forge by default. This version of SimpleITK is being installed: Simpleitk :: Anaconda.org. To install the simpleitk channel version, you can specify the channel when adding the package. CondaPkg.add("SimpleITK"; channel="simpleitk"). However, I think the conda-forge version is more up to date.

1 Like
8

Thank you so much, this is so simple and helpful. Saves me many more days of random debugging

2 Likes