Myujth

I'm trying to get data from a website using async in python. As an example I used this code (under A Better Coroutine Example): https://www.blog.pythonlibrary.org/2016/07/26/python-3-an-intro-to-asyncio/

Now this works fine, but it writes the binary chunks to a file and I don't want it in a file. I want the resulting data directly. But I currently have a list of coroutine objects which I can not get the data out of.

The code:

# -*- coding: utf-8 -*-
import aiohttp
import asyncio
import async_timeout

async def fetch(session, url):
 with async_timeout.timeout(10):
 async with session.get(url) as response:
 return await response.text()


async def main(loop, urls):
 async with aiohttp.ClientSession(loop=loop) as session:
 tasks = [fetch(session, url) for url in urls]
 await asyncio.gather(*tasks)
 return tasks

# time normal way of retrieval
if __name__ == '__main__':
 urls = [a list of urls..]

 loop = asyncio.get_event_loop()
 details_async = loop.run_until_complete(main(loop, urls))

Thanks

The problem is in return tasks at the end of main(), which is not present in the original article. Instead of returning the coroutine objects (which are not useful once passed to asyncio.gather), you should be returning the tuple returned by asyncio.gather, which contains the results of running the coroutines in correct order. For example:

async def main(loop, urls):
 async with aiohttp.ClientSession(loop=loop) as session:
 tasks = [fetch(session, url) for url in urls]
 results = await asyncio.gather(*tasks)
 return results

Now loop.run_until_complete(main(loop, urls)) will return a tuple of texts in the same order as the URLs.