Does PythonCall block all `@async` routines?

General Usage
, ,
1

I am trying to write a simple timing function. To avoid waiting for too long, I made it stop waiting and return Inf at some point. I achieved it this way:

function time_function(f, input...; maxtime=1)
    c = Channel{Float64}(2) do ch # Watchdog
        sleep(maxtime)
        put!(ch, Inf)
    end
    @async begin                  # Worker
        dt = @elapsed f(input...)
        isopen(c) && put!(c, dt)
    end
    return take!(c)
end

The point being, it doesn’t work with Python functions for some reason:

using PythonCall
t = pyimport("time")
time_function(sleep, 0.1)      # 0.1
time_function(sleep, 10)       # Inf
time_function(t.sleep, 0.1)    # 0.1
time_function(t.sleep, 10)     # 10 - why?

Seems that the Python functions are tunning in the main thread and are await-ed regardless of @async.

How can I avoid this? I’m aware of this problem, but in my case I do not use Python’s asyncio at all.

2

With @async you are tryjng to share a thread with Python, but Python does not know that. Consider using @spawn instead while making sure Threads.nthreads() is greater than 1.

3

Unfortunately, this didn’t help. However, I eventually created a separate implementation for Python functions:

pyexec("""
import threading
import time
import ctypes

def _async_raise(tid, exctype):
    '''Abort the thread'''
    if not isinstance(tid, int):
        tid = tid.ident
    res = ctypes.pythonapi.PyThreadState_SetAsyncExc(ctypes.c_long(tid), ctypes.py_object(exctype))
    if res == 0:
        raise ValueError("Invalid thread id")
    elif res > 1:
        # If it returns a number greater than one
        # We're in trouble, and we try to revert the effect
        ctypes.pythonapi.PyThreadState_SetAsyncExc(ctypes.c_long(tid), None)
        raise SystemError("PyThreadState_SetAsyncExc failed")""", Main)

@pyexec ```
def py_measure_time(func, maxtime, *args):
    def wrapper():
        func(*args)

    thread = threading.Thread(target=wrapper)
    thread.start()
    start_time = time.time()
    thread.join(timeout=maxtime)
    if thread.is_alive():
        _async_raise(thread, SystemExit)
        return float("Inf")
    else:
        return time.time() - start_time``` => py_measure_time
time_function(f::Py, args...; maxtime=1) = 
    pyconvert(Number, py_measure_time(f, maxtime, args...))