How to use asyncio in Python
asyncio is a module that provides tools for implementing asynchronous programming. It was introduced in Python 3.3 and has since become a popular choice for creating and managing asynchronous tasks.
The module is available in the standard library, and can therefore be imported and used directly without extra installations.
import the asyncio module
Example
import asyncio
print(asyncio)
Output:
<module 'asyncio' from '/app/.heroku/python/lib/python3.11/asyncio/__init__.py'>
Basic usage
Asynchronous programming makes it possible to execute multiple pieces of code in a non-blocking manner. The basic idea is to execute a task and when a delay is encountered, execution can switch to another task instead of blocking and waiting for the delay to finish.
Asynchronous tasks are created and managed using the async and await keywords in addition to the various functions defined in the asyncio module. Consider the following example:
Example
basic asyncio usage
#import the module
import asyncio
#define asynchronous tasks
async def task1():
#prints even numbers from 0 to 9
for i in range(10):
if i%2 ==0:
print(i)
await asyncio.sleep(0.0001)#cause a small delay
async def task2():
#prints odd numbers from 0 to 9
for i in range(10):
if i%2 == 1:
print(i)
await asyncio.sleep(0.0001) # cause a small delay
async def main():
print('Started:')
await asyncio.gather(task1(), task2())
print('Finished!')
asyncio.run(main())
Output:
Started: 0 1 2 3 4 5 6 7 8 9 Finished!
In the above snippet, we:
imported the
asynciomodule.Defined two asynchronous tasks(corourines),
task1, andtask2.task1prints even numbers from0to9whiletask2prints odd numbers in the same range. Inside each coroutine theasyncio.sleep()call simulates a delay in the execution of the coroutine, this makes the execution to move to the other coroutine.We defined the
maincoroutine, then awaitedtask1andtask2inside it. Theasyncio.gather()function combines multiple coroutines into a single awaitable object.We executed the
main()coroutine using theasyncio.run()function.The result is that
task1andtask2are executed in parallel, as you can tell from the output.
The above example may look somehow complicated but this is the most complete basic example of a practical asynchronous program I can think of.
In the following sections, we will explore the two keywords and the various asyncio functions and what each does.
async/await keywords
Python provides the async and await keywords for creating coroutines. Coroutines are functions that can be executed asynchronously. To be more specific, coroutines are functions in which execution can be suspended and then resumed later on.
async defines a function as an asynchronous coroutine.awaitsuspends the running coroutine until another coroutine is fully executed.
The async keyword
Defining coroutine function is much like defining regular functions. We simply put the async keyword before def to tell the interpreter that this is a coroutine function
Example
Define a coroutine function
async def greet(name):
print('Hello ' + name)
Output:
Unlike in a regular function, the statement in a coroutine do not get executed immediately when the coroutine is called. Instead, a coroutine object is created.
Example
calling a coroutine
async def greet(name):
print('Hello ' + name)
coro = greet('Jane')#This does not execute the statements in the coroutine
print(coro)#a coroutine object is created
Output:
<coroutine object greet at 0x7fa671749f20>
To actually execute the statements in a given coroutine, you have to await it in what is referred to as an event loop. Python provides a simpler way to do this through the asyncio.run() function.
The asyncio.run() function takes a single coroutine as the argument and then automatically performs tasks that a programmer would be required to handle manually, such as scheduling, creating an event loop, running the event loop, etc
Example
import asyncio
async def greet(name):
print('Hello ' + name)
coro = greet('Jane')#This does not execute the statements in the coroutine
asyncio.run(coro)#run the coroutine
Output:
Hello Jane
The await keyword
Consider what you would do if you wanted to run a coroutine inside of another coroutine. One might be tempted to use the asyncio.run() function inside the coroutine, but this would automatically lead to a RunTimeError exception.
Example
import asyncio
async def greet(name):
print('Hello ' + name)
async def main():
asyncio.run(greet('John'))
coro = main()
asyncio.run(coro)#runtime error
Output:
RuntimeError: asyncio.run() cannot be called from a running event loop
To run a coroutine inside of another coroutine, you need to use await keyword. The keyword has the following syntax:
Syntax:
await <coroutine>
await makes the running coroutine to be suspended until the awaited <coroutine> is completely executed.
Example
import asyncio
async def greet(name):
print('Hello ' + name)
async def main():
await greet('John')
coro = main()
asyncio.run(coro)
Output:
Hello John
Running coroutines simultaneously
The primary goal of asynchronous programming is to execute multiple tasks in a non-blocking manner. This means that a task does not have to wait until another task is fully executed in order to start.
To run multiple tasks in parallel you would be required to manually interact with the event loop as we will see later. But again, Python offers a simpler way to do this. The process is as illustrated below:
Create asynchronous coroutines.
Combine and schedule them using
asyncio.gather()functionAwait the coroutine object returned by
asyncio.gather()in another coroutine eg.main().Run main using
asyncio.run()function.
Let us look again at the example that we started with.
Example
#import the module
import asyncio
#define asynchronous tasks
async def task1():
#prints even numbers from 0 to 9
for i in range(10):
if i%2 ==0:
print(i)
await asyncio.sleep(0.0001)#cause a small delay
async def task2():
#prints odd numbers from 0 to 9
for i in range(10):
if i%2 == 1:
print(i)
await asyncio.sleep(0.0001) # cause a small delay
async def main():
print('Started:')
await asyncio.gather(task1(), task2())
print('Finished!')
asyncio.run(main())
Output:
Started: 0 1 2 3 4 5 6 7 8 9 Finished!
If you observe the above output, you will see that the two function are actually being executed at the same time as the output is emitted alternately. Or are they?
One thing that you should keep in mind is that true simultaneity is not achievable with asynchronous programming. In reality, the two tasks are not being executed simultaneously. The small delay caused by asyncio.sleep(0.0001) is enough to make the execution to switch between the two tasks.
Example
#import the module
import asyncio
#define asynchronous tasks
async def print_nums():
for n in range(5):
print(n)
await asyncio.sleep(0.0001)#cause a small delay
async def print_letters():
#prints odd numbers from 0 to 9
for l in 'abcde':
print(l)
await asyncio.sleep(0.0001) # cause a small delay
async def main():
print('Started:')
await asyncio.gather(print_nums(), print_letters())
print('Finished!')
asyncio.run(main())
Output:
Started: 0 a 1 b 2 c 3 d 4 e Finished!
This is essentially how asynchronous programming works, when a delay is encountered in a running coroutine, the execution is passed to the next coroutine in the schedule. This can be especially useful when dealing with tasks that may lead to blockage such as I/O bound tasks. As it allows the program to continue executing other tasks instead of being blocked until the task is completed.
The event loop
We have been talking about the event loop without formally introducing it.
In this section we will manually interact with event loops to run manage asynchronous tasks. We will see what the asyncio.run() function is doing in the background.
Creating an event loop
The new_event_loop() function in the asyncio module creates and returns an event loop.
Example
create a loop
import asyncio
loop = asyncio.new_event_loop()
print(loop)
print(loop.is_running())
print(loop.is_closed())
loop.close()
print(loop.is_closed())
Output:
<_UnixSelectorEventLoop running=False closed=False debug=False> False False True
In the above example, we created an event loop then used some methods to check the state of the loop. The is_running() method checks whether a loop is currently running. The is_closed() method checks whether a loop is closed.
You should always remember to call the close() method after you are done using the loop.
run a single coroutine in an event loop
The loop.run_until_complete() can be used to manually execute a one-off coroutine in an event loop.
Example
import asyncio
#the coroutine
async def task():
print('Hello, World!')
loop = asyncio.new_event_loop() #create the loop
loop.run_until_complete(task()) #run the coroutine
loop.close()#close the loop
Output:
Hello, World!
Run multiple coroutines in an event loop
When multiple coroutines have to be executed simultaneously, we will be required to create an event loop, schedule the tasks in the event loop, the execute them. The following snippet shows a basic example:
Example
import asyncio
#the tasks
async def print_nums(num):
for n in range(num):
print(n)
await asyncio.sleep(0.0001)#simulate
async def print_letters(up_to):
for letter in range(97,ord(up_to)):
print(chr(letter))
await asyncio.sleep(0.0001)#simulate delay
#create an event loop
loop = asyncio.new_event_loop()
#schedule the tasks for execution
loop.create_task(print_nums(5))
loop.create_task(print_letters('f'))
#get all the tasks scheduled in the loop
tasks = asyncio.all_tasks(loop = loop)
#create an awaitable for the tasks
group = asyncio.gather(*tasks)
#run the loop
loop.run_until_complete(group)
#close the loop
loop.close()
Output:
0 a 1 b 2 c 3 d 4 e