Non-blocking asyncio calls with PyCall

General Usage
,
1

Hello everyone,

I am currently trying to integrate asyncronous Python functions that use asyncio into Julia’s coroutine framework. My problem is similar to the following example:

using PyCall
asyncio = pyimport("asyncio")

py"""
async def sleep_well(time):
        print("Falling asleep...")
        await $asyncio.sleep(time)
        print("Wake up!")
"""

loop = asyncio.get_event_loop()
@async loop.run_until_complete(py"sleep_well"(3))

Since PyCall uses a simple ccall under the hood, the last line is of course not running asynchronously.
Is there a way to change that, for example by calling Julia’s yield() from Python and notifying Julia when the work is done?

I found this old question which goes in the same direction but I could not manage to get any of the proposed approaches working.

2

A basic issue is that asyncio expects its own event loop, which is different from the libuv event loop used by Julia.

Apparently it is possible to hack asyncio to use a libuv event loop — see https://github.com/MagicStack/uvloop — and maybe this could be used to integrate it with Julia, but you’d have to dig into the guts of the uvloop implementation.

3

Thank you for the fast response!
I just gave the uvloop package a first (naive) try:

using PyCall
asyncio = pyimport("asyncio")
uvloop = pyimport("uvloop")

py"""
async def sleep_well(time):
        print("Falling asleep...")
        await $asyncio.sleep(time)
        print("Wake up!")
"""

loop = uvloop.new_event_loop()
asyncio.set_event_loop(loop) # Edit: This line should instead be asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
@async loop.run_until_complete(py"sleep_well"(3))

loop turned into a uvloop now but (as expected) the last line is still not running asynchronously. I am not really familiar with libuv, but I assume that instead of using uvloop.new_event_loop() the handle to Julia’s uvloop (Base.eventloop()?) needs to be converted to a uvloop object and passed to asyncio.

Could this be the right way or am I getting something wrong already?

4

Julia doesn’t have an “event-loop object” — that is not part of libuv. The point of the uvloop, as I understand it, is to take libuv and wrap it in Python objects that mimic asyncio’s APIs.

You will have to look at how uvloop is implemented, and how Julia’s event loop is implemented, and translate the concepts as needed. This will take some digging.

2 Likes
5

Alright, then I start digging. Thank you again!

6

IIUC asyncio’s event loop is pluggable by design. For example, there is asyncio.set_event_loop for setting a current event loop for the current OS thread.

@mkosch You might have discovered it already but asyncio.AbstractEventLoop seems to be a good starting point. Presumably you need to implement this interface using PyCall for a full interop.

If you just want to a quick fake solution, maybe you can do something like ipyjulia_hacks — ipyjulia_hacks 0 documentation (it just polls in background)

1 Like