How does the memory model of pyjulia work?

General Usage
1

I am using Julia from Python via pyjulia which has a how it works.

But I want to understand if there are performance implications. E.g. if I have a numpy array in python and use it in Julia; is Julia making a copy or using the array in place?

Vice versa, when I use a vector created in Julia in python, is it using the same block of memory, or is it making a copy?

2

I am pretty sure no copy is made. To check it just mutate an array on one side and see if the mutation is present on the other side.

1 Like
3

I remember having issues with unnecessary copies, Python NumPy array is not modified inside Julia · Issue #385 · JuliaPy/pyjulia · GitHub

4

From the pycall docs:

From Julia to Python

Assuming you have NumPy installed (true by default if you use Conda), then a Julia a::Array of NumPy-compatible elements is converted by PyObject(a) into a NumPy wrapper for the same data , i.e. without copying the data. Julia arrays are stored in column-major order, and since NumPy supports column-major arrays this is not a problem.

However, the default ordering of NumPy arrays created in Python is row-major, and some Python packages will throw an error if you try to pass them column-major NumPy arrays. To deal with this, you can use PyReverseDims(a) to pass a Julia array as a row-major NumPy array with the dimensions reversed . For example, if a is a 3x4x5 Julia array, then PyReverseDims(a) passes it as a 5x4x3 NumPy row-major array (without making a copy of the underlying data).

A Vector{UInt8} object in Julia, by default, is converted to a Python bytearray object. If you want a bytes object instead, you can use the function pybytes(a) .

From Python to Julia

Multidimensional NumPy arrays ( ndarray ) are supported and can be converted to the native Julia Array type, which makes a copy of the data.

Alternatively, the PyCall module also provides a new type PyArray (a subclass of AbstractArray ) which implements a no-copy wrapper around a NumPy array (currently of numeric types or objects only). Just use PyArray as the return type of a pycall returning an ndarray , or call PyArray(o::PyObject) on an ndarray object o . (Technically, a PyArray works for any Python object that uses the NumPy array interface to provide a data pointer and shape information.)

Conversely, when passing arrays to Python, Julia Array types are converted to PyObject types without making a copy via NumPy, e.g. when passed as pycall arguments.

There is more information here .

2 Likes