Python Programming / Python Asyncio

Async HTTP with aiohttp

Make many HTTP requests concurrently using aiohttp and asyncio with proper timeouts and error handling.

2 min readupdated 6 Oct 2026

Why aiohttp

The requests library is blocking.

For asyncio, you typically use:

aiohttp

It supports non-blocking HTTP requests.

Basic GET

Python

import asyncio
import aiohttp
 
 
async def main():
    timeout = aiohttp.ClientTimeout(total=10)
 
    async with aiohttp.ClientSession(timeout=timeout) as session:
        async with session.get("https://api.github.com") as resp:
            data = await resp.json()
            print(resp.status)
            print(list(data.keys())[:5])
 
 
asyncio.run(main())

Fetch many URLs concurrently

Python

import asyncio
import aiohttp
 
 
async def fetch(session: aiohttp.ClientSession, url: str) -> int:
    async with session.get(url) as resp:
        await resp.read()
        return resp.status
 
 
async def main():
    urls = [
        "https://httpbin.org/delay/1",
        "https://httpbin.org/delay/1",
        "https://httpbin.org/delay/1",
    ]
 
    timeout = aiohttp.ClientTimeout(total=5)
 
    async with aiohttp.ClientSession(timeout=timeout) as session:
        statuses = await asyncio.gather(*(fetch(session, u) for u in urls), return_exceptions=True)
        print(statuses)
 
 
asyncio.run(main())

Notes

Use timeouts.
Handle non-200 responses.
Avoid hitting API rate limits (use semaphores).

Why HTTP is the ideal asyncio workload

A request spends nearly all its life doing nothing — the bytes are in flight, and the CPU is idle. That waiting is exactly what an event loop can overlap:

diagram Diagram mermaid

  flowchart LR
  subgraph SEQ["requests, one at a time"]
    S1["send"] --> W1["wait 0.2s"] --> R1["read"] --> S2["send"] --> W2["wait 0.2s"] --> R2["read"]
  end
  subgraph ASY["aiohttp with gather"]
    A1["send all 12"] --> AW["all 12 wait together
0.2s total"] --> AR["read all 12"]
  end

Measured against a local server whose endpoint sleeps 0.20 s, 12 requests:

approach	time	speedup
`requests.get` in a loop	2.53 s	1.00×
`requests.Session` in a loop	2.47 s	1.03×
`aiohttp` + `gather`	0.22 s	11.6×

A local server was used deliberately: the delay is known and fixed, so the figures say something about the client rather than about someone else's network.

Note the middle row. Reusing a Session saves the TCP and TLS handshake, which is real on a remote host but nearly free on localhost — 1.03× here. Connection reuse is not what buys the 11.6×; overlapping the waiting is.

The shape of the code

Python

import asyncio, aiohttp
 
async def fetch(session, url):
    async with session.get(url) as resp:      # releases the loop while waiting
        return resp.status, await resp.json()
 
async def main(urls):
    async with aiohttp.ClientSession() as session:   # ONE session for all requests
        return await asyncio.gather(*[fetch(session, u) for u in urls])
 
if __name__ == "__main__":
    results = asyncio.run(main([f"http://localhost:8000/slow/{i}" for i in range(12)]))

Two details carry most of the benefit:

One ClientSession for the whole batch. It owns the connection pool. Creating one per request throws that away and is the most common aiohttp mistake.
async with on the response. It releases the connection back to the pool. Without it the pool drains and later requests queue behind nothing.

See it move

Twelve requests, a server that takes 0.20 s each. Drag the concurrency limit and watch the batches form — the total is simply how many rounds you need.

sketch Concurrency limit sets the number of rounds p5.js

With 12 requests at 0.2s each, a limit of 3 needs 4 rounds and a limit of 6 needs 2. Measured times match that arithmetic almost exactly.

Measured against the local server:

limit	rounds needed	predicted	measured
`Semaphore(3)`	4	0.80 s	0.83 s
`Semaphore(6)`	2	0.40 s	0.42 s
no limit (12)	1	0.20 s	0.22 s

The arithmetic is the whole model: time ≈ ceil(N / limit) × latency. Overhead is roughly 20–30 ms on top.

Python

async def fetch_all(urls, limit=6):
    sem = asyncio.Semaphore(limit)
    async with aiohttp.ClientSession() as session:
        async def one(url):
            async with sem:                       # at most `limit` inside at a time
                async with session.get(url) as r:
                    return r.status, await r.json()
        return await asyncio.gather(*[one(u) for u in urls])

Check yourself

quizCheck yourself

12 requests to an endpoint that takes 0.2 s: a requests loop took 2.53 s and aiohttp with gather took 0.22 s. Where does the gain come from?
Show answer
B — the waiting overlaps, since each request releases the loop while its bytes are in flight — Almost all of a request is waiting. One thread can hold twelve waits at once, so the total approaches a single round trip rather than twelve.
Reusing requests.Session for the same 12 local requests gave only 1.03x. Why so little?
Show answer
B — a Session helps by skipping TCP and TLS setup, which is nearly free over localhost — Connection reuse removes handshake cost, which matters over a real network and barely registers locally. It is not what produces the 11.6x — overlapping the waiting is.
With 12 URLs at 0.2 s each and asyncio.Semaphore(3), roughly how long should the batch take?
Show answer
B — about 0.8 s, because 12 / 3 gives 4 sequential rounds — A semaphore of 3 permits three in flight at a time, so the work forms 4 rounds of 0.2 s. Predicted 0.80 s, measured 0.83 s.
Why create one aiohttp.ClientSession for a whole batch rather than one per request?
Show answer
B — the session owns the connection pool, so a session per request discards connection reuse and setup cost — A ClientSession holds the connector and its pool. Building one per request recreates that machinery every time and is the most common aiohttp mistake.

🧪 Try It Yourself

Exercise 1 – Your First Coroutine

exerciseYour turn

script.py

Output

Press Run to see what your code prints.

Exercise 2 – await asyncio.sleep

exerciseYour turn

script.py

Output

Press Run to see what your code prints.

Exercise 3 – Gather Two Coroutines

exerciseYour turn

script.py

Output

Press Run to see what your code prints.

Async HTTP with aiohttp

Why aiohttp

Basic GET

Fetch many URLs concurrently

Notes

Why HTTP is the ideal asyncio workload

The shape of the code

See it move

Check yourself

🧪 Try It Yourself

Exercise 1 – Your First Coroutine

Exercise 2 – await asyncio.sleep

Exercise 3 – Gather Two Coroutines

Was this page helpful?