# How to Efficiently Index PyArrays?

**URL:** https://discourse.julialang.org/t/how-to-efficiently-index-pyarrays/69785
**Category:** Performance
**Tags:** question, pycall, python
**Created:** [October 14, 2021, 10:04pm UTC](https://discourse.julialang.org/t/how-to-efficiently-index-pyarrays/69785 "2021-10-14T22:04:28Z")
**Posts on this page:** 6
**Page:** 2

<div class="post-metadata">

### Author: ![cjdoris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cjdoris/32/213133_2.png) [@cjdoris](https://discourse.julialang.org/u/cjdoris)
#### Post date: [October 16, 2021, 11:29am UTC](https://discourse.julialang.org/t/how-to-efficiently-index-pyarrays/69785/21 "2021-10-16T11:29:28Z")

</div>

> [@goerch](#):
>
> ```julia
> function index_test(pyarray::PyArray{UInt8, 3})
> s = 0
> for i in 1:size(array, 1)
> for j in 1:size(array, 2)
> for k in 1:size(array, 3)
> s += array[i, j, k]
> end
> end
> end
> s
> end
> 
> ```

The argument is `pyarray` not `array`. So you are timing reading a global variable and not touching PyCall at all.

---

<div class="post-metadata">

### Author: ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)
#### Post date: [October 16, 2021, 11:56am UTC](https://discourse.julialang.org/t/how-to-efficiently-index-pyarrays/69785/22 "2021-10-16T11:56:07Z")

</div>

Embarrassing mistake on my side. Corrected test:

```julia
using PyCall

function index_test(pyarray::PyArray{UInt8, 3})
    s = 0
    for i in 1:size(pyarray, 1)
        for j in 1:size(pyarray, 2)
            for k in 1:size(pyarray, 3)
                s += pyarray[i, j, k]
            end
        end
    end
    s
end

function from_python(pyarray::PyArray{UInt8, 3})
    return convert(Array{UInt8, 3}, pyarray)
end

function index_test(array::Array{UInt8, 3})
    s = 0
    for i in 1:size(array, 1)
        for j in 1:size(array, 2)
            for k in 1:size(array, 3)
                s += array[i, j, k]
            end
        end
    end
    s
end

function to_python(array::Array{UInt8, 3})
    PyArray(PyObject(array))
end

array = Array{UInt8, 3}(undef, (250, 250, 3))
@btime index_test($array)
@btime to_python($array)
pyarray = PyArray(PyObject(array))
@btime index_test($pyarray)
@btime from_python($pyarray)
println()

```

with result

```julia
  87.800 μs (0 allocations: 0 bytes)
  1.820 μs (20 allocations: 1.16 KiB)
  161.300 μs (0 allocations: 0 bytes)
  469.700 μs (2 allocations: 183.20 KiB)

```

Everything looks fine!

---

<div class="post-metadata">

### Author: ![rewindturtle](https://avatars.discourse-cdn.com/v4/letter/r/0ea827/32.png) [@rewindturtle](https://discourse.julialang.org/u/rewindturtle)
#### Post date: [October 16, 2021, 7:39pm UTC](https://discourse.julialang.org/t/how-to-efficiently-index-pyarrays/69785/23 "2021-10-16T19:39:07Z")

</div>

My bad. Tested it again. It is 50us faster with the `Array{Float32, 2}(undef, 92, 120)`. I originally tested it with `Matrix{Float32}(undef, 92, 120)`.

---

<div class="post-metadata">

### Author: ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)
#### Post date: [October 16, 2021, 7:41pm UTC](https://discourse.julialang.org/t/how-to-efficiently-index-pyarrays/69785/24 "2021-10-16T19:41:27Z")

</div>

those are exactly the same

---

<div class="post-metadata">

### Author: ![rewindturtle](https://avatars.discourse-cdn.com/v4/letter/r/0ea827/32.png) [@rewindturtle](https://discourse.julialang.org/u/rewindturtle)
#### Post date: [October 16, 2021, 7:56pm UTC](https://discourse.julialang.org/t/how-to-efficiently-index-pyarrays/69785/25 "2021-10-16T19:56:42Z")

</div>

I am new to Julia so sorry if I seem so naive.

---

<div class="post-metadata">

### Author: ![rewindturtle](https://avatars.discourse-cdn.com/v4/letter/r/0ea827/32.png) [@rewindturtle](https://discourse.julialang.org/u/rewindturtle)
#### Post date: [October 16, 2021, 7:59pm UTC](https://discourse.julialang.org/t/how-to-efficiently-index-pyarrays/69785/26 "2021-10-16T19:59:01Z")

</div>

I tried the index test with different versions of PyCall. Using v1.90.0, only 11 allocations are made. Using any earlier version gives me the error:

```julia
ERROR: LoadError: InitError: could not load symbol "Py_CompileString":
The specified procedure could not be found. 
Stacktrace:
 [1] macro expansion
   @ C:\Users\user\.julia\packages\PyCall\WcrLS\src\exception.jl:81 [inlined]
 [2] pyeval_(s::String, globals::PyCall.PyDict{String, PyCall.PyObject, false}, locals::PyCall.PyDict{String, PyCall.PyObject, false}, input_type::Int64, fname::String)
   @ PyCall C:\Users\user\.julia\packages\PyCall\WcrLS\src\pyeval.jl:20
 [3] pyeval_
   @ C:\Users\user\.julia\packages\PyCall\WcrLS\src\pyeval.jl:17 [inlined]
 [4] __init__ ()
   @ PyCall C:\Users\user\.julia\packages\PyCall\WcrLS\src\pyinit.jl:131
 [5] _include_from_serialized(path::String, depmods::Vector{Any})
   @ Base .\loading.jl:696
 [6] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
   @ Base .\loading.jl:782
 [7] _require(pkg::Base.PkgId)
   @ Base .\loading.jl:1020
 [8] require(uuidkey::Base.PkgId)
   @ Base .\loading.jl:936
 [9] require(into::Module, mod::Symbol)
   @ Base .\loading.jl:923
during initialization of module PyCall
in expression starting at C:\Users\user\PycharmProjects\juliaProject\arcade.jl:7

```

[Previous page](https://discourse.julialang.org/t/how-to-efficiently-index-pyarrays/69785.md?page=1)
