I am trying to use SimpleITK via PythonCall inside a Pluto notebook. I want to load a DICOM image and then convert it to a numpy array. The code to do this is straightforward and works in pure python, but with PythonCall this produces an error.
using CondaPkg, DICOM, PythonCall
CondaPkg.add("SimpleITK")
CondaPkg.add("numpy")
sitk = pyimport("SimpleITK")
np = pyimport("numpy")
root_path = "/Users/daleblack/Library/CloudStorage/GoogleDrive-djblack@uci.edu/My Drive/Datasets/perfusion/limb"
dicom_path = joinpath(root_path, "DICOM")
fixed_dicom_dir = joinpath(dicom_path, "01")
function load_dicom_sitk(path)
series_ids = sitk.ImageSeriesReader.GetGDCMSeriesIDs(path)
series_file_names = sitk.ImageSeriesReader.GetGDCMSeriesFileNames(path, series_ids[0])
reader = sitk.ImageSeriesReader()
reader.SetFileNames(series_file_names)
image = reader.Execute()
image = sitk.Cast(image, sitk.sitkFloat32)
end
fixed_image = load_dicom_sitk(fixed_dicom_dir)
sitk.GetArrayFromImage(fixed_image)
With this error
Python: ImportError: NumPy not available.
pythrow()@err.jl:94
PythonCall@err.jl:10[inlined]
PythonCall@object.jl:210[inlined]
var"#pycall#59"(::Base.Pairs{Symbol, Union{}, Tuple{}, @NamedTuple{}}, ::typeof(PythonCall.pycall), ::PythonCall.Py, ::PythonCall.Py)@object.jl:228
Main@object.jl:218[inlined]
Main@Py.jl:341[inlined]
Main@Local: 1[inlined]
Any idea how to get this to work? I want to be able to convert a Julia array to an sitk object and to do this I need to go through numpy I think
It also errors in the other direction, trying to convert julia array to sitk Image
function load_dcm_array(dcm_data::Vector{DICOM.DICOMData})
return array = cat(
[dcm_data[i][tag"Pixel Data"] for i in 1:length(dcm_data)]...; dims=3
)
end
dcms_jl = DICOM.dcmdir_parse(fixed_dicom_dir);
fixed_image_jl = load_dcm_array(dcms_jl);
fixed_image_pyarr = PyArray(fixed_image_jl);
sitk.GetImageFromArray(fixed_image_pyarr)
Python: ImportError: Numpy not available.
pythrow()@err.jl:94
PythonCall@err.jl:10[inlined]
PythonCall@object.jl:210[inlined]
var"#pycall#59"(::Base.Pairs{Symbol, Union{}, Tuple{}, @NamedTuple{}}, ::typeof(PythonCall.pycall), ::PythonCall.Py, ::PythonCall.PyArray{Int16, 3, true, true, Int16})@object.jl:228
Main@object.jl:218[inlined]
Main@Py.jl:341[inlined]
Main@Local: 1[inlined]
I figured out that I get the same error if I run this in a Jupyter notebook using Python kernel. And after restarting the kernel, the error goes away. Maybe something like that is occurring in Julia?
When you import PythonCall.jl
at the start, an environment with Python 3.11 is created:
Package Version Build Channel Size
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
Install:
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
+ libstdcxx-ng 11.4.0 h4dcbe23_1 conda-forge Cached
+ _libgcc_mutex 0.1 conda_forge conda-forge Cached
+ ld_impl_linux-64 2.40 h41732ed_0 conda-forge Cached
+ ca-certificates 2023.7.22 hbcca054_0 conda-forge Cached
+ libgomp 13.2.0 h807b86a_1 conda-forge Cached
+ _openmp_mutex 4.5 2_gnu conda-forge Cached
+ libgcc-ng 13.2.0 h807b86a_1 conda-forge Cached
+ openssl 3.1.3 hd590300_0 conda-forge Cached
+ libzlib 1.2.13 hd590300_5 conda-forge Cached
+ libffi 3.4.2 h7f98852_5 conda-forge Cached
+ bzip2 1.0.8 h7f98852_4 conda-forge Cached
+ ncurses 6.4 hcb278e6_0 conda-forge Cached
+ libuuid 2.38.1 h0b41bf4_0 conda-forge Cached
+ libexpat 2.5.0 hcb278e6_1 conda-forge Cached
+ xz 5.2.6 h166bdaf_0 conda-forge Cached
+ libnsl 2.0.0 h7f98852_0 conda-forge Cached
+ libsqlite 3.43.0 h2797004_0 conda-forge Cached
+ tk 8.6.12 h27826a3_0 conda-forge Cached
+ readline 8.2 h8228510_1 conda-forge Cached
+ tzdata 2023c h71feb2d_0 conda-forge Cached
+ python 3.11.5 hab00c5b_0_cpython conda-forge Cached
+ wheel 0.41.2 pyhd8ed1ab_0 conda-forge Cached
+ setuptools 68.2.2 pyhd8ed1ab_0 conda-forge Cached
+ pip 23.2.1 pyhd8ed1ab_0 conda-forge Cached
Then, when SimpleITK
is added to the conda environment, a different Python version is installed
Package Version Build Channel Size
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
Install:
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
+ expat 2.5.0 hcb278e6_1 conda-forge Cached
+ zlib 1.2.13 hd590300_5 conda-forge Cached
+ python_abi 3.9 4_cp39 conda-forge Cached
+ jpeg 9e h0b41bf4_3 conda-forge Cached
+ libpng 1.6.39 h753d276_0 conda-forge Cached
+ libdeflate 1.10 h7f98852_0 conda-forge Cached
+ libgfortran4 7.5.0 h14aa051_20 conda-forge Cached
+ jbig 2.1 h7f98852_2003 conda-forge Cached
+ libwebp-base 1.3.2 hd590300_0 conda-forge Cached
+ lz4-c 1.9.3 h9c3ff4c_1 conda-forge Cached
+ lerc 3.0 h9c3ff4c_0 conda-forge Cached
+ eigen 3.4.0 h4bd325d_0 conda-forge Cached
+ libgfortran-ng 7.5.0 h14aa051_20 conda-forge Cached
+ zstd 1.5.2 h8a70e8d_1 conda-forge Cached
+ fftw 3.3.8 nompi_hfc0cae8_1114 conda-forge Cached
+ hdf5 1.10.6 nompi_h3c11f04_101 conda-forge Cached
+ libtiff 4.3.0 h542a066_3 conda-forge Cached
+ libitk 5.1.2 h86d312b_0 conda-forge Cached
+ libitk-devel 5.1.2 h9c3ff4c_0 conda-forge Cached
+ simpleitk 2.0.2 py39h156e488_1 conda-forge Cached
Reinstall:
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
o libstdcxx-ng 11.4.0 h4dcbe23_1 conda-forge Cached
o wheel 0.41.2 pyhd8ed1ab_0 conda-forge Cached
o setuptools 68.2.2 pyhd8ed1ab_0 conda-forge Cached
o pip 23.2.1 pyhd8ed1ab_0 conda-forge Cached
Downgrade:
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
- python 3.11.5 hab00c5b_0_cpython conda-forge Cached
+ python 3.9.18 h0755675_0_cpython conda-forge Cached
I have only briefly looked at how PythonCall.jl
links to Python, but I think that since it is already imported, PythonCall.jl
is pointing to the original Python version (3.11).
julia> PythonCall.C.CTX.lib_path
"/tmp/jl_0SO6uT/.CondaPkg/env/lib/libpython3.11.so.1.0"
One fix would be to create the environment before importing PythonCall.jl
. This can be done with a CondaPkg.toml
file, or the following modification to your code:
using CondaPkg
CondaPkg.add("SimpleITK", resolve=false)
CondaPkg.add("numpy")
using DICOM, PythonCall
sitk = pyimport("SimpleITK")
np = pyimport("numpy")
root_path = "/Users/daleblack/Library/CloudStorage/GoogleDrive-djblack@uci.edu/My Drive/Datasets/perfusion/limb"
dicom_path = joinpath(root_path, "DICOM")
fixed_dicom_dir = joinpath(dicom_path, "01")
function load_dicom_sitk(path)
series_ids = sitk.ImageSeriesReader.GetGDCMSeriesIDs(path)
series_file_names = sitk.ImageSeriesReader.GetGDCMSeriesFileNames(path, series_ids[0])
reader = sitk.ImageSeriesReader()
reader.SetFileNames(series_file_names)
image = reader.Execute()
image = sitk.Cast(image, sitk.sitkFloat32)
end
fixed_image = load_dicom_sitk(fixed_dicom_dir)
sitk.GetArrayFromImage(fixed_image)
3 Likes
cjdoris
September 20, 2023, 7:52am
5
@tylerjthomas9 is exactly right.
This is a known (but undocumented) gotcha - similar to how you shouldnβt install Julia packages after importing anything (it can lead to incompatible packages) - only in this case itβs the python interpreter itself which is incompatible, with more disastrous consequences.
Suggestions of how to help avoid this welcome.
3 Likes
Palli
September 20, 2023, 11:17am
6
I.e. 3.9, but itβs unclear to me why. SimpleITK supports 3.11, and in Conda, i.e. conda-forge, 3.11 is support (as opposed to Anaconda, which supports at least 3.10 seemingly).
Python 3.9 should still work (with or without Julia).
Palli:
I.e. 3.9, but itβs unclear to me why. SimpleITK supports 3.11, and in Conda supports at least 3.10 seemingly, should be fixable to 3.11.
What version of Julia are you using? PythonCall.jl
adds libstdcxx-ng
bounds in conda to ensure that the Python packages are compatible with Julia. This means that the latest Python package versions may not be installed when using older Julia versions since there would be incompatibilities. With Julia v1.9.3
, your original code runs for me without any errors.
Additionally, CondaPkg.jl
uses the channel conda-forge
by default. This version of SimpleITK
is being installed: Simpleitk :: Anaconda.org . To install the simpleitk
channel version, you can specify the channel when adding the package. CondaPkg.add("SimpleITK"; channel="simpleitk")
. However, I think the conda-forge
version is more up to date.
1 Like
Thank you so much, this is so simple and helpful. Saves me many more days of random debugging
2 Likes
EDIT: Sorry this doesnβt belong in this thread. I opened up a separate topic for this here
Resurrecting this because I am still dealing with a similar issue. Just adding numpy and/or torch results in an error when doing this in Pluto
I have tried with and without resolve
begin
using Pkg; Pkg.activate(temp = true)
Pkg.add("CondaPkg")
using CondaPkg
# CondaPkg.add("numpy", resolve = false)
CondaPkg.add("numpy")
CondaPkg.add("torch")
end
failed process: Process(`/Users/daleblack/.julia/artifacts/5767a45b62528e0be4c40dbae502ad3e132f1d62/bin/micromamba -r /Users/daleblack/.julia/scratchspaces/0b3b1443-0f03-428d-bdfb-f27f9c1191ea/root create -y -p /private/var/folders/cc/2fhyrfnd7x39rgjk39v38nk40000gn/T/jl_l84GJ5/.CondaPkg/env --override-channels --no-channel-priority "numpy[version='*']" "torch[version='*']" -c conda-forge`, ProcessExited(1)) [1]
Stack trace
Here is what happened, the most recent locations are first:
pipeline_error @ process.jl:565
run(::Cmd; wait::Bool) @ process.jl:480
run(::Cmd) @ process.jl:477
_run(io::IO, cmd::Cmd, args::Any; flags::Any) @ resolve.jl:398
_resolve_conda_install(io::Any, conda_env::Any, specs::Any, channels::Any; create::Any) @ resolve.jl:299
resolve(; force::Bool, io::IO, interactive::Bool, dry_run::Bool) @ resolve.jl:547
resolve() @ resolve.jl:405
add(pkgs::AbstractVector; resolve::Any, file::Any, kw...) @ deps.jl:222
add(pkgs::AbstractVector) @ deps.jl:215
add(pkg::Union{CondaPkg.ChannelSpec, CondaPkg.PipPkgSpec, CondaPkg.PkgSpec}; kw...) @ deps.jl:226
add(pkg::AbstractString; version::Any, channel::Any, build::Any, kw...) @ deps.jl:315
This cell: line 7
CondaPkg.add("numpy")
CondaPkg.add("torch")
end
Does anyone see what is still going wrong?
Julia Version 1.10.2
Commit bd47eca2c8a (2024-03-01 10:14 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: macOS (arm64-apple-darwin22.4.0)
CPU: 8 Γ Apple M1
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-15.0.7 (ORCJIT, apple-m1)
Threads: 4 default, 0 interactive, 2 GC (on 4 virtual cores)
Environment:
JULIA_REVISE_WORKER_ONLY = 1
False alarm, it looks like itβs just called βpytorchβ when using anaconda and βtorchβ when using pip
This works correctly:
begin
using Pkg; Pkg.activate(temp = true)
Pkg.add("CondaPkg")
using CondaPkg
CondaPkg.add("numpy")
CondaPkg.add("pytorch")
end