Understanding asyncio in python

Question

I've been having a hard time understanding Python's asyncio module and how to not block on asynchronous calls. For instance, given this code snippet:

import aiohttp
import asyncio
import async_timeout

async def fetch(session, url):
    with async_timeout.timeout(10):
        async with session.get(url) as response:
            return await response.text()

async def main(loop):
    print(1)
    async with aiohttp.ClientSession(loop=loop) as session:
        html = await fetch(session, 'http://python.org')
        print(html)
    print(2)

loop = asyncio.get_event_loop()
loop.run_until_complete(main(loop))

I would expect, similar to how this would work in Javascript, the output to be

1
2
<!doctype html>
<...>

Instead, the function prints 1, blocks until it gets back the html, then prints 2. Why does it block, and how / can I avoid the blocking? Thanks for your help.

Community · Accepted Answer · 2017-05-23 12:09:44Z

Your problem is this line:

html = await fetch(session, 'http://python.org')

this means - wait until getting a response from fetch, Or in other word's it's blocking the current coroutine until the fetch coroutine return's the response text.

To make things simpler:

The fetch operation by itself isn't blocking, only the await.
The await is blocking only for the current coroutine, not the thread.
Like vincent said, calling the operation by itself doesn't do anything (we don't start the fetching process)

from asyncio documentation

Calling a coroutine does not start its code running – the coroutine object returned by the call doesn’t do anything until you schedule its execution. There are two basic ways to start it running: call await coroutine or yield from coroutine from another coroutine (assuming the other coroutine is already running!), or schedule its execution using the ensure_future() function or the AbstractEventLoop.create_task() method.

And a fix (for getting the desired output):

import asyncio
import aiohttp
import async_timeout


async def fetch(session, url):
    with async_timeout.timeout(10):
        async with session.get(url) as response:
            return await response.text()


async def main(loop):
    print(1)
    async with aiohttp.ClientSession(loop=loop) as session:
        future = asyncio.ensure_future(fetch(session, 'http://python.org')) # we start the fetch process
        print(2)
        html = await future # we wait for getting the fetch response
        print(html)


loop = asyncio.get_event_loop()
loop.run_until_complete(main(loop))

Calling the fetch coroutine doesn't schedule the fetch operation, and it doesn't return a future. You have to use asyncio.ensure_future in order to achieve that.
Thanks for correcting my answer, I edited my code and added additional explanation.
Thanks, this explanation was very helpful. So, if I did want to start the coroutine without blocking execution of the rest of the function... I would effectively need to await an asyncio.sleep(0) after the asyncio.ensure_future?? Is there a better way to do that?
using asyncio.ensure_future start the coroutine and dosn't block the rest of the function
@etlsh I guess what I mean is that, if I add an await asyncio.sleep(0) just after the asyncio.ensure_future, it will actually start executing the coroutine before 2 prints. Otherwise, it won't even begin until after the 2 prints, correct?

dekassert · Accepted Answer · 2017-02-23 21:48:44Z

To understanding asyncio in python 3.6 async def style I suggest you to run this simple example:

import asyncio
import time

def write(msg):
    print(msg, flush=True)

async def say1():
    await asyncio.sleep(1)
    write("Hello 1!")

async def say2():
    await asyncio.sleep(1)
    write("Hello 2!")

write("start")
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.gather(
    say1(),
    say2()
))
write("exit")

loop.close()

Source: https://maketips.net/tip/146/parallel-execution-of-asyncio-functions See link for detailed explanations of this code with call diagram .

Collectives™ on Stack Overflow

Understanding asyncio in python

2 Answers 2

6 Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

Comments

Related