Juliacall / PythonCall array conversion error

[greatpet]

I’m using the Python package juliacall to call my Julia code. I encounter sporadic errors from lines that look like this:

import numpy as np
from juliacall import Main as jl
jl.seval("import mypackage")
jl_mypackage = jl.seval("mypackage")
a0, b0 = jl_mypackage.somefunction(some_argument) # returns two arrays
a, b = np.asarray(a0), np.asarray(b0)

The last two lines are actually inside a loop, and I get sporadic errors every ~100K calls, with error messages like:

     13 a0, b0 = mypackage.somefunction(some_argument)
---> 14 a, b = np.asarray(a), np.asarray(b)

File ~/.julia/packages/PythonCall/L4cjh/src/JlWrap/array.jl:338, in __array__(self, dtype)
    336 if not (hasattr(arr, "__array_interface__") or hasattr(arr, "__array_struct__")):
    337     # the first attempt collects into an Array
--> 338     arr = self._jl_callmethod($(pyjl_methodnum(pyjlarray_array__array)))
    339     if not (hasattr(arr, "__array_interface__") or hasattr(arr, "__array_struct__")):
    340         # the second attempt collects into a PyObjectArray

JuliaError: MethodError: no method matching pyjlarray_array__array(::Nothing)
The function `pyjlarray_array__array` exists, but no method is defined for this combination of argument types.

Closest candidates are:
  pyjlarray_array__array(!Matched::AbstractArray)
   @ PythonCall ~/.julia/packages/PythonCall/L4cjh/src/JlWrap/array.jl:277

The same errors occur if I use np.array, instead of np.asarray, to wrap the Julia arrays while making a copy. My Julia code has type annotation to ensure that jl_mypackage.somefunction(some_argument) returns a tuple of two numerical arrays.

The only relevant result I found on Google is an unanswered github issue from another user. The same error message, no method matching pyjlarray_array__array(::Nothing), appears near the end of his post:

Juliacall with SB3 for RL in WSL bings to segmetation fault problems

Unlike the other post, I’m using Linux instead of WSL, but curiously, I’ve also used the same Python packages (PyTorch and SB3) when I run into this problem. But as far as I understand, the essential part of the problem is captured by the few lines of code I posted at the beginning, not related to the other packages, as my code is essentially standalone for providing data.

P.S. Now I have a cleaner demonstration of the problem in a new post:

Juliacall / PythonCall array conversion crash (now with MWE)

Here’s the Python code which extracts a tuple of two Julia arrays from the juliacall interface and converts them to numpy arrays. The code is an infinite loop, but crashes sporadically after a few million iterations with segfault: # juliacall_problem1.py from juliacall import Main as jl import numpy as np counter = 0 while True: counter += 1 try: a, b = jl.seval("(fill(0f0, 3, 8, 8), fill(true, 3, 8, 8))") c, d = np.array(a), np.array(b) except: print(f"Fail…