Slicing arrays with a Colon via PythonCall

General Usage
1

This works in Julia

julia> rand(3,5)[:, 1]

and this works in Python

>>> import numpy as np
>>> np.random.rand(3,5)[:, 1]

but this doesn’t work with PythonCall

julia> using PythonCall
julia> np = pyimport("numpy")
julia> np.random.rand(3,5)[:, 1]
ERROR: Python: IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices
Python stacktrace: none
Stacktrace:
 [1] pythrow()
   @ PythonCall.Core ~/.julia/packages/PythonCall/S5MOg/src/Core/err.jl:92
 [2] errcheck
   @ ~/.julia/packages/PythonCall/S5MOg/src/Core/err.jl:10 [inlined]
 [3] pygetitem(x::PythonCall.Core.Py, k::Tuple{Colon, Int64})
   @ PythonCall.Core ~/.julia/packages/PythonCall/S5MOg/src/Core/builtins.jl:171
 [4] getindex(::PythonCall.Core.Py, ::Function, ::Int64)
   @ PythonCall.Core ~/.julia/packages/PythonCall/S5MOg/src/Core/Py.jl:293
 [5] top-level scope
   @ REPL[6]:1

Is there any way to get colons passing through correctly to Python here? I’d really prefer natural colons, rather than some ugly call like

np.random.rand(3,5)[pyslice(nothing), 1]

I had claimed that you could just use python modules from Julia with almost the same syntax as in python, but this is a big glaring exception to that claim.

2

You need to manually convert the array to a PyArray.

julia> np = pyimport("numpy");

julia> A = np.random.rand(3,5)
Python:
array([[0.57881241, 0.87821956, 0.00514754, 0.08800188, 0.14813928],
       [0.95910626, 0.69120523, 0.81646325, 0.16623818, 0.99664291],
       [0.40534225, 0.01465493, 0.39096707, 0.40985706, 0.80201301]])

julia> PyArray(A)
3×5 PyArray{Float64, 2}:
 0.578812  0.87822    0.00514754  0.0880019  0.148139
 0.959106  0.691205   0.816463    0.166238   0.996643
 0.405342  0.0146549  0.390967    0.409857   0.802013

julia> PyArray(A)[:,3]
3-element Vector{Float64}:
 0.005147539092155484
 0.81646324920764
 0.39096706544477833
3

Okay, but I would put that in the same category of unnaturalness and ugliness as

np.random.rand(3,5)[pyslice(nothing), 1]

The point is that it really seems like the Julia object Colon() represented by : in

julia> np.random.rand(3,5)[:, 1]

should just work — maybe by being translated to pyslice(nothing); maybe by some other means. But I’m failing to understand what pygetitem is doing, so I don’t understand why it’s not working. Is this just a bug, a feature that hasn’t been implemented, or a real limitation?

4

The short answer is that there is no conversion method Py(::Colon) defined for : yet.

It would probably make sense to convert it to pyslice(nothing) as you suggest.

I’ve not thought about it too hard, but was put off a bit because you can’t really emulate Python’s full slicing syntax (start:stop etc) in Julia. Firstly because in Python it is special indexing syntax whereas in Julia it is just a way to get a range. And secondly because the natural conversion for a Julia range is a Python range, which is not a valid index for a python list (which only allows indexing by int and slice). So slicing would be a bit inconsistent.

That said, some other libraries such as numpy do allow indexing by range (I think?) so we’d actually get more consistency there.

Another consideration is what happens over on the Python side in JuliaCall. If you do juliacall.Base.Colon() then currently you’ll get a wrapped : which can be used to index a Julia array from Python. If we add the above conversion rule, you’ll instead get slice(None) which cannot be used as a Julia array index. To fix this we’d also need to add a conversion rule for slice(None) to : which I guess is ok? This consideration will be irrelevant in PythonCall v1 (in the works) which will always return wrapped Julia objects from JuliaCall.

1 Like