How to properly use asyncio run_coroutine_threadsafe function?

Question

I am trying to understand asyncio module and spend about one hour with run_coroutine_threadsafe function, I even came to the working example, it works as expected, but works with several limitations.

First of all I do not understand how should I properly call asyncio loop in main (any other) thread, in the example I call it with run_until_complete and give it a coroutine to make it busy with something until another thread will not give it a coroutine. What are other options I have?

What are situations when I have to mix asyncio and threading (in Python) in real life? Since as far as I understand asyncio is supposed to take place of threading in Python (due to GIL for not IO ops), if I am wrong, do not be angry and share your suggestions.

Python version is 3.7/3.8

import asyncio
import threading
import time


async def coro_func():
    return await asyncio.sleep(3, 42)


def another_thread(_loop):
    coro = coro_func()  # is local thread coroutine which we would like to run in another thread

    # _loop is a loop which was created in another thread

    future = asyncio.run_coroutine_threadsafe(coro, _loop)
    print(f"{threading.current_thread().name}: {future.result()}")
    time.sleep(15)
    print(f"{threading.current_thread().name} is Finished")


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    main_th_cor = asyncio.sleep(10)
    # main_th_cor  is used to make loop busy with something until another_thread will not send coroutine to it
    print("START MAIN")
    x = threading.Thread(target=another_thread, args=(loop, ), name="Some_Thread")
    x.start()
    time.sleep(1)
    loop.run_until_complete(main_th_cor)
    print("FINISH MAIN")

user4815162342 · Accepted Answer · 2020-02-09 09:40:28Z

First of all I do not understand how should I properly call asyncio loop in main (any other) thread, in the example I call it with run_until_complete and give it a coroutine to make it busy with something until another thread will not give it a coroutine. What are other options I have?

This is a good use case for loop.run_forever(). The loop will run and serve the coroutines you submit using run_coroutine_threadsafe. (You can even submit such coroutines from multiple threads in parallel; you never need to instantiate more than one event loop.)

You can stop the loop from a different thread by calling loop.call_soon_threadsafe(loop.stop).

What are situations when I have to mix asyncio and threading (in Python) in real life?

Ideally there should be none. But in the real world, they do crop up; for example:

When you are introducing asyncio into an existing large program that uses threads and blocking calls and cannot be converted to asyncio all at once. run_coroutine_threadsafe allows regular blocking code to make use of asyncio.
When you are dealing with older "async" APIs which use threads under the hood and call the user-supplied APIs from other threads. There are many examples, such as Python's own multiprocessing.
When you need to call blocking functions that have no async equivalent from asyncio - e.g. CPU-bound functions, legacy database drivers, things like that. This is not a use case for run_coroutine_threadsafe, here you'd use run_in_executor, but it is another example of mixing threads and asyncio.

Collectives™ on Stack Overflow

How to properly use asyncio run_coroutine_threadsafe function?

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related