### Author:  [@paulmelis](https://discourse.julialang.org/u/paulmelis)
#### Post date: [September 1, 2020, 1:35pm UTC](https://discourse.julialang.org/t/pyjulia-passing-numpy-array-from-python-side/45894/1 "2020-09-01T13:35:44Z")
I’m looking into extending Python with Julia code through PyJulia, particularly allocating numpy arrays in Python, passing them to Julia (without copying) to do some processing and then use the result in Python again. This seems to be the reverse of what many other users are doing (i.e. using `PyCall` from Julia to call into Python), so maybe what I want is not supported.
But looking at the docs for PyCall, specifically [GitHub - JuliaPy/PyCall.jl: Package to call Python functions from the Julia language](https://github.com/JuliaPy/PyCall.jl#arrays-and-pyarray), seems to suggest it is possible to pass a NumPy array from Python to Julia without copying:
> Alternatively, the PyCall module also provides a new type `PyArray` (a subclass of `AbstractArray` ) which implements a no-copy wrapper around a NumPy array (currently of numeric types or objects only). Just use `PyArray` as the return type of a `pycall` returning an `ndarray` , or call `PyArray(o::PyObject)` on an `ndarray` object `o` . (Technically, a `PyArray` works for any Python object that uses the NumPy array interface to provide a data pointer and shape information.)
However, the referenced `PyArray` type lives on the Julia side so I can’t call it on the Python side. Using a Julia function with signature `fn(x::PyArray)` also doesn’t seem to do automatic conversion, as I get a `no method matching fn(::Array{Float32,2})` error when called from Python with:
```julia
melis@juggle 15:33:~$ cat fn.jl
function fn(x::PyArray)
println("array size: $(size(x))");
println("max element: $(maximum(x))")
println("min element: $(minimum(x))")
x[1,1] = 123
return 2x
end
melis@juggle 15:33:~$ cat callit.py
from julia.api import Julia
from julia import Main
jl = Julia(compiled_modules=False)
jl.eval('include("fn.jl")')
import numpy as np
x = np.array([[1,2,3], [4,5,6]], dtype=np.float64)
res = Main.fn(x)
melis@juggle 15:34:~$ python callit.py
Traceback (most recent call last):
File "callit.py", line 11, in
res = Main.fn(x)
RuntimeError: Julia exception: MethodError: no method matching fn(::Array{Float64,2})
Closest candidates are:
fn(!Matched::PyArray) at /home/melis/concepts/blender-julia/fn.jl:1
Stacktrace:
[1] #invokelatest#1 at ./essentials.jl:710 [inlined]
[2] invokelatest(::Any, ::Any) at ./essentials.jl:709
[3] _pyjlwrap_call(::Function, ::Ptr{PyCall.PyObject_struct}, ::Ptr{PyCall.PyObject_struct}) at /home/melis/.julia/packages/PyCall/zqDXB/src/callback.jl:28
[4] pyjlwrap_call(::Ptr{PyCall.PyObject_struct}, ::Ptr{PyCall.PyObject_struct}, ::Ptr{PyCall.PyObject_struct}) at /home/melis/.julia/packages/PyCall/zqDXB/src/callback.jl:49
```
Any other ways to make this work?
Edit: word
---
### Author:  [@jishnub](https://discourse.julialang.org/u/jishnub)
#### Post date: [September 1, 2020, 2:07pm UTC](https://discourse.julialang.org/t/pyjulia-passing-numpy-array-from-python-side/45894/2 "2020-09-01T14:07:38Z")
Why not remove the type specification on `x` in the `fn` definition? Just define `fn(x)` so that it works for any array-like object.
---
### Author:  [@paulmelis](https://discourse.julialang.org/u/paulmelis)
#### Post date: [September 1, 2020, 2:10pm UTC](https://discourse.julialang.org/t/pyjulia-passing-numpy-array-from-python-side/45894/3 "2020-09-01T14:10:28Z")
I don’t think that guarantees the array will not be copied
---
### Author:  [@lungben](https://discourse.julialang.org/u/lungben)
#### Post date: [September 1, 2020, 3:02pm UTC](https://discourse.julialang.org/t/pyjulia-passing-numpy-array-from-python-side/45894/4 "2020-09-01T15:02:09Z")
You could test if there is a copy e.g. by defining a numpy array with 1 billion entries (8 GB for float64) and passing it to Julia, while looking at your system memory consumption.
The most simple calling of Julia functions on Numpy arrays seems to make indeed a copy of them.
Test code:
```julia
import julia
jl = julia.Julia()
import numpy as np
my_sum = jl.eval("my_sum(x) = sum(x)")
small_a = np.random.randn(100)
my_sum(small_a) # compile method
a = np.random.randn(1_000_000_000) # 8 GB
np.sum(a)
my_sum(a) # memory consumtion increases significantly, indicating a copy
```
---
### Author:  [@paulmelis](https://discourse.julialang.org/u/paulmelis)
#### Post date: [September 1, 2020, 3:04pm UTC](https://discourse.julialang.org/t/pyjulia-passing-numpy-array-from-python-side/45894/5 "2020-09-01T15:04:32Z")
> [@lungben](#):
>
> You could test if there is a copy e.g. by defining a numpy array with 1 billion entries (8 GB for float64) and passing it to Julia, while looking at your system memory consumption.
When I alter the array on the Julia side I also don’t see the change on the Python side when the call is done.
---
### Author:  [@polpastells](https://discourse.julialang.org/u/polpastells)
#### Post date: [April 21, 2021, 7:36am UTC](https://discourse.julialang.org/t/pyjulia-passing-numpy-array-from-python-side/45894/6 "2021-04-21T07:36:49Z")
I’m having exactly this problem, did you find a solution?
---
### Author:  [@paulmelis](https://discourse.julialang.org/u/paulmelis)
#### Post date: [April 21, 2021, 7:54am UTC](https://discourse.julialang.org/t/pyjulia-passing-numpy-array-from-python-side/45894/7 "2021-04-21T07:54:51Z")
Just checked my test code from months ago. Using `unsafe_wrap` I apparently managed to get something working:
```julia
# alter_array.jl
function fn(a)
a[:] .= 9
end
function fn(addr, length)
a = unsafe_wrap(Array{UInt32}, Ptr{UInt32}(addr), length)
fn(a)
end
```
```julia
# t_alter_array.py
import numpy
import julia
from julia.api import Julia
jl = Julia()
from julia import Main
jl.eval('include("alter_array.jl")')
a = numpy.array([1, 2, 3, 4, 5], 'uint32')
print(a)
Main.fn(a)
print(a)
addr = a.ctypes.data
length = a.shape[0]
Main.fn(addr, length)
print(a)
```
```julia
$ python t_alter_array.py
[1 2 3 4 5]
[1 2 3 4 5]
[9 9 9 9 9]
```
Note that the call to `Main.fn(a)` does _not_ result in the array being altered, it only works by passing the array address and length, i.e. `Main.fn(addr, length)`.
---
### Author:  [@polpastells](https://discourse.julialang.org/u/polpastells)
#### Post date: [April 21, 2021, 12:04pm UTC](https://discourse.julialang.org/t/pyjulia-passing-numpy-array-from-python-side/45894/8 "2021-04-21T12:04:35Z")
Thanks for the quick response. This worked.
---
### Author:  [@stevengj](https://discourse.julialang.org/u/stevengj)
#### Post date: [April 21, 2021, 12:17pm UTC](https://discourse.julialang.org/t/pyjulia-passing-numpy-array-from-python-side/45894/9 "2021-04-21T12:17:33Z")
> [@paulmelis](#):
>
> ```julia
> addr = a.ctypes.data
> length = a.shape[0]
>
> ```
Don’t do this. Just [use `pyfunction`](https://github.com/JuliaPy/PyCall.jl#calling-julia-from-python) to declare the argument-type conversions that you want. That’s what it’s for.
Something like `pyfn = Main.PyCall.pyfunction(Main.fn, Main.PyCall.PyArray)` should tell it to pass the argument of `pyfn` as a `PyArray`, which is a no-copy wrapper.
---
### Author:  [@paulmelis](https://discourse.julialang.org/u/paulmelis)
#### Post date: [April 21, 2021, 12:50pm UTC](https://discourse.julialang.org/t/pyjulia-passing-numpy-array-from-python-side/45894/10 "2021-04-21T12:50:07Z")
> [@stevengj](#):
>
> Don’t do this. Just [use `pyfunction`](https://github.com/JuliaPy/PyCall.jl#calling-julia-from-python) to declare the argument-type conversions that you want. That’s what it’s for.
>
> Something like `pyfn = Main.PyCall.pyfunction(Main.fn, Main.PyCall.PyArray)` should tell it to pass the argument of `pyfn` as a `PyArray` , which is a no-copy wrapper.
Trying that now I get an error I remember seeing before:
```julia
$ cat t_alter_array.py
import numpy
import julia
from julia.api import Julia
jl = Julia()
from julia import Main
jl.eval('include("alter_array.jl")')
a = numpy.array([1, 2, 3, 4, 5], 'uint32')
print(a)
pyfn = Main.PyCall.pyfunction(Main.fn, Main.PyCall.PyArray)
pyfn(a)
print(a)
$ py t_alter_array.py
[1 2 3 4 5]
Traceback (most recent call last):
File "/home/melis/concepts/blender-julia-test/test/t_alter_array.py", line 21, in
pyfn = Main.PyCall.pyfunction(Main.fn, Main.PyCall.PyArray)
File "/home/melis/.local/lib/python3.9/site-packages/julia/core.py", line 176, in __getattr__
return self.__try_getattr(name)
File "/home/melis/.local/lib/python3.9/site-packages/julia/core.py", line 191, in __try_getattr
if self._julia.isdefined(realname):
File "/home/melis/.local/lib/python3.9/site-packages/julia/core.py", line 645, in isdefined
raise ValueError(
ValueError: `julia.isdefined(name)` requires at least one dot in `name`.
```
I initially reported a similar error in the PyCall.jl repo, but you replied back then that is was a pyjulia error: [Exception when trying to navigate Main.PyCall module · Issue #813 · JuliaPy/PyCall.jl · GitHub](https://github.com/JuliaPy/PyCall.jl/issues/813). Which I then subsequently reported in the pyjulia repo where it has been dormant marked “bug” ever since: [Exception when trying to acces submodules of Main · Issue #414 · JuliaPy/pyjulia · GitHub](https://github.com/JuliaPy/pyjulia/issues/414).
So unless I’m missing something setting up the `pyfunction` as you suggest currently doesn’t seem possible. And that was perhaps the reason I tried the dirty hack passing the array address 😉
---
### Author:  [@polpastells](https://discourse.julialang.org/u/polpastells)
#### Post date: [April 21, 2021, 1:53pm UTC](https://discourse.julialang.org/t/pyjulia-passing-numpy-array-from-python-side/45894/11 "2021-04-21T13:53:55Z")
I have no idea of what I’m doing, but `pyfn = Main.eval("pyfunction(fn, PyArray)")` seems to work for me.
On top of that it preserves the numpy data-types and dimensions correctly.
---
### Author:  [@paulmelis](https://discourse.julialang.org/u/paulmelis)
#### Post date: [April 22, 2021, 8:59am UTC](https://discourse.julialang.org/t/pyjulia-passing-numpy-array-from-python-side/45894/12 "2021-04-22T08:59:06Z")
That’s an even better workaround, nice
---
### Author:  [@cjdoris](https://discourse.julialang.org/u/cjdoris)
#### Post date: [April 23, 2021, 6:34am UTC](https://discourse.julialang.org/t/pyjulia-passing-numpy-array-from-python-side/45894/13 "2021-04-23T06:34:54Z")
Plug: You might also like to try my package PythonCall.jl and it’s companion juliacall (which are similar to PyCall.jl and pyjulia) because there all mutable values are passed without copying by default.
---
### Author:  [@NLeureka](https://discourse.julialang.org/u/NLeureka)
#### Post date: [May 25, 2021, 12:40am UTC](https://discourse.julialang.org/t/pyjulia-passing-numpy-array-from-python-side/45894/14 "2021-05-25T00:40:57Z")
Could you show a specific example from Python to Julia, such as calling functions written in Python and returning to Julia? There is no similar example in the document. It’s not friendly for beginners of Julia.