Passing numpy arrays between python and Julia

General Usage
1

I am still new to Julia, so just wanted to incrementally test out some features. One thing I wanted to try was accelerating some python code I have by offloading it to Julia.

The first scenario is simple. So I have a dictionary of model settings that I want to pass to Julia. Then after running the model in julia–a time series simulation–I wanted to return a multidimensional array from julia back to python. To be precise by multi-dimensional I mean a time series that has a set of variables in the columns, the rows indexed by the timestep, and finally the depth dimension indexed by complete run of the simulation (where I might run 100 complete simulations).

So like I said, I pass the model parameters as a dictionary and then get back a numpy array.

I was looking at PyCall.jl to do this. The documentation says:

Multidimensional arrays exploit the NumPy array interface for conversions between Python and Julia. By default, they are passed from Julia to Python without making a copy, but from Python to Julia a copy is made; no-copy conversion of Python to Julia arrays can be achieved with the PyArray type below.

However, I did not see an example of this feature implemented in actual code. So do I need to create an empty array and pass it as a reference to the Julia PyCall function, or do I need to do any special handling of these types of requests. Does anyone have an example code to do this?

Oops, forgot to mention that I looked around for some other posts on this topic, because I figured it was not unique. The closest I found was this post below, but I did not see any code in it as an example.

Thanks.

2 Likes
2

If you just want to “get back a numpy array”, you don’t need to do anything:

julia> using PyCall

julia> py"""
       def printflags(xs):
           print(xs.flags)
       """

julia> pyprintflags = py"printflags"
PyObject <function printflags at 0x7fe387b5cea0>

julia> pyprintflags(ones(2, 3))
  C_CONTIGUOUS : False
  F_CONTIGUOUS : True
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False

Observe that OWNDATA flag is False; Python/Numpy does not own the data but Julia does in this case.

Or, an example a bit more closer to what you describe is

julia> py"""
       jlones = $ones
       """

julia> py"""
       printflags(jlones(2, 3))
       """
  C_CONTIGUOUS : False
  F_CONTIGUOUS : True
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False
1 Like
3

Hey @tkf this is helpful. I had not seen this approach to interrogating the calls, so that is nice. Do you know the corresponding python code to call into Julia? Like if I am in python and I wanted to pass the array to a Julia function.

4

A minimal example is something like this

julia> using PyCall

julia> py"""
       import numpy
       xs = numpy.ones((2, 3))
       """

julia> xs = PyArray(py"xs"o)
2×3 PyArray{Float64,2}:
 1.0  1.0  1.0
 1.0  1.0  1.0

julia> xs[1, 1] = 2
2

julia> py"""
       print(xs)
       """
[[2. 1. 1.]
 [1. 1. 1.]]

Notice the o in py"xs"o; this tells PyCall to not auto-convert/copy the Numpy array xs. This is then converted to PyArray manually.

To pass a Julia function to Python, you need to use pyfunction:

julia> printtype(x) = println(typeof(x))
printtype (generic function with 1 method)

julia> pyjlprinttype = pyfunction(printtype, PyArray)
PyObject <PyCall.jlwrap PyCall.FuncWrapper{Tuple{PyArray},typeof(printtype)}(printtype, Dict{Symbol,Any}())>

julia> py"""
       jlprinttype = $pyjlprinttype
       jlprinttype(xs)
       """
PyArray{Float64,2}
3 Likes