Making HTTP Requests in Python with Requests and HTTPX

Maria
Python
02 Oct, 2026

Almost every Python project ends up talking to something over HTTP: a payment provider, an internal service, a weather API, a webhook. The standard library can do it with urllib.request, but nearly everyone reaches for a third-party client instead because the ergonomics are so much better.

For years that client was Requests. HTTPX arrived later with a deliberately similar API plus async support, HTTP/2, and stricter defaults. This post covers both side by side: making requests, sending JSON and forms, handling timeouts and errors properly, reusing connections, retrying, going async, streaming large downloads, and testing code that calls APIs. By the end you'll know how to use either one well and which to choose for a new project.

All examples call httpbin.org, a free service that echoes back what you send, so you can run them as-is.

Installing

python -m pip install requests httpx

Both work on Python 3.13+. HTTPX's optional HTTP/2 support needs an extra: pip install "httpx[http2]".

Your First GET Request

The two libraries look almost identical for simple calls:

# get_requests.py
import requests

response = requests.get(
    "https://httpbin.org/get",
    params={"q": "python", "page": 2},
    headers={"Accept": "application/json"},
    timeout=10,
)
print(response.url)
print(response.status_code, response.ok, response.headers["content-type"])
print(response.json()["args"])

https://httpbin.org/get?q=python&page=2
200 True application/json
{'page': '2', 'q': 'python'}

# get_httpx.py
import httpx

response = httpx.get("https://httpbin.org/get", params={"q": "python"})
print(response.url, response.status_code, response.is_success, response.http_version)

https://httpbin.org/get?q=python 200 True HTTP/1.1

What each piece does:

params builds the query string and URL-encodes values for you. Never build query strings with f-strings; special characters will break them. A list value like {"tags": ["a", "b"]} becomes tags=a&tags=b.
headers sets request headers. Header lookups on the response are case-insensitive, so response.headers["Content-Type"] and ["content-type"] both work.
response.json() parses the body as JSON and raises an error if it isn't valid JSON.
response.text gives the decoded body as a string, and response.content gives raw bytes.

Notice that the httpx example has no timeout. That's the first real difference between the libraries, and it's important enough to get its own section.

Timeouts: The Setting You Must Not Skip

Requests has no default timeout. If the server accepts the connection and then never responds, requests.get() waits forever, and a worker process hangs with it. Always pass timeout:

import requests

try:
    requests.get("https://httpbin.org/delay/5", timeout=(3.05, 1))
except requests.Timeout as exc:
    print("Timed out:", type(exc).__name__)

Timed out: ReadTimeout

A single number sets both the connect and read timeouts. A tuple sets them separately: here 3.05 seconds to establish the connection and 1 second between bytes of the response. Note that the read timeout is the maximum gap between bytes, not a limit on the total download time.

HTTPX defaults to a 5 second timeout for everything, which is a much safer default. You can configure it finely with httpx.Timeout:

import httpx

timeout = httpx.Timeout(10.0, connect=3.0)
print(timeout)

Timeout(connect=3.0, read=10.0, write=10.0, pool=10.0)

pool is how long to wait for a free connection from the client's connection pool. Pass timeout=None to disable timeouts entirely, but think hard before you do.

Sending Data: JSON, Forms, and Files

JSON Bodies

Most modern APIs take JSON. Use the json= argument, which serializes your data and sets Content-Type: application/json:

import requests

response = requests.post(
    "https://httpbin.org/post",
    json={"name": "Ada", "role": "admin"},
    timeout=10,
)
body = response.json()
print(body["json"], body["headers"]["Content-Type"])

{'name': 'Ada', 'role': 'admin'} application/json

The httpx call is the same: httpx.post(url, json={...}).

Form Data

HTML-style form posts use data=, which sends application/x-www-form-urlencoded:

response = requests.post("https://httpbin.org/post", data={"name": "Ada"}, timeout=10)
print(response.json()["form"], response.json()["headers"]["Content-Type"])

{'name': 'Ada'} application/x-www-form-urlencoded

Mixing these up is a common bug: passing a dict to data= when the API expects JSON will send form encoding, and the server will usually respond with a 400 or 415 error.

File Uploads

Multipart uploads use files=. The value can be a file object or a tuple of (filename, file, content_type):

import httpx

with open("report.pdf", "rb") as fh:
    response = httpx.post(
        "https://httpbin.org/post",
        files={"upload": ("report.pdf", fh, "application/pdf")},
    )
print(list(response.json()["files"]))

Open files in binary mode ("rb"). Requests uses the same files= argument.

Handling Errors

Neither library raises an exception for a 4xx or 5xx response by default. A 404 is a successful HTTP exchange that happens to carry bad news. You decide what's an error:

import requests

response = requests.get("https://httpbin.org/status/404", timeout=10)
print(response.status_code, response.ok)

try:
    response.raise_for_status()
except requests.HTTPError as exc:
    print("HTTPError:", exc)

404 False
HTTPError: 404 Client Error: NOT FOUND for url: https://httpbin.org/status/404

In HTTPX, raise_for_status() raises httpx.HTTPStatusError, which carries both exc.request and exc.response:

import httpx

try:
    httpx.get("https://httpbin.org/status/500").raise_for_status()
except httpx.HTTPStatusError as exc:
    print("Status:", exc.response.status_code)
except httpx.RequestError as exc:
    print("Network problem:", type(exc).__name__, exc.request.url)

There are two families of failure, and it helps to handle them separately:

Failure	Requests	HTTPX
Base class for everything	`requests.RequestException`	`httpx.HTTPError`
Server returned an error status (after `raise_for_status`)	`requests.HTTPError`	`httpx.HTTPStatusError`
Couldn't connect (DNS, refused)	`requests.ConnectionError`	`httpx.ConnectError`
Timed out	`requests.Timeout`	`httpx.TimeoutException`
Any transport-level failure	`requests.RequestException`	`httpx.RequestError`

Transport errors (couldn't connect, timed out) are often worth retrying. Most 4xx errors aren't: a 401 or 422 will fail the same way every time.

Redirects

Requests follows redirects automatically (for every method except HEAD) and records the hops in response.history. HTTPX does not follow redirects unless you ask:

import httpx
import requests

r = requests.get("https://httpbin.org/redirect/1", timeout=10)
print(r.status_code, [h.status_code for h in r.history])

r = httpx.get("https://httpbin.org/redirect/1")
print(r.status_code)

r = httpx.get("https://httpbin.org/redirect/1", follow_redirects=True)
print(r.status_code, r.url)

200 [302]
302
200 https://httpbin.org/get

If you port code from Requests to HTTPX and suddenly see 301 or 302 responses, this is why. Set follow_redirects=True on the call or on the client.

Sessions and Clients: Reuse Connections

The top-level functions (requests.get, httpx.get) open a new connection for every call. When you make more than a couple of requests to the same host, use a requests.Session or an httpx.Client. They keep connections alive in a pool (skipping repeated TCP and TLS handshakes), persist cookies, and let you set defaults once.

# session_requests.py
import requests

with requests.Session() as session:
    session.headers.update({"Authorization": "Bearer abc123"})
    r = session.get("https://httpbin.org/headers", timeout=10)
    print(r.json()["headers"]["Authorization"])

HTTPX's Client goes further with a base_url, default timeout, and default headers:

# client_httpx.py
import httpx

with httpx.Client(
    base_url="https://httpbin.org",
    headers={"User-Agent": "tidewave/1.0"},
    timeout=10.0,
) as client:
    r = client.get("/headers")
    print(r.json()["headers"]["User-Agent"])
    r = client.post("/post", json={"a": 1})
    print(r.json()["json"])

tidewave/1.0
{'a': 1}

Use a with block, or call client.close(), so pooled connections are released. In a long-running app, create one client at startup and share it; creating a new client per request throws away the pooling.

Authentication

Both libraries support HTTP Basic auth with a tuple:

import requests

r = requests.get("https://httpbin.org/basic-auth/user/pass", auth=("user", "pass"), timeout=10)
print(r.status_code, r.json())

200 {'authenticated': True, 'user': 'user'}

For bearer tokens and API keys, set a header on the session or client as shown above. Keep the secret in an environment variable, not in source code.

Retries

Neither library retries by default, but both make it easy to add.

With Requests, mount an HTTPAdapter configured with urllib3's Retry:

# retries_requests.py
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

retry = Retry(
    total=3,
    backoff_factor=0.5,
    status_forcelist=[429, 500, 502, 503, 504],
    allowed_methods=["GET", "PUT", "DELETE"],
)
session = requests.Session()
session.mount("https://", HTTPAdapter(max_retries=retry))

response = session.get("https://httpbin.org/get", timeout=10)

This retries on connection errors and on the listed status codes, with exponential backoff, and respects Retry-After headers. allowed_methods matters: it leaves POST out by default because retrying a non-idempotent request could, for example, charge a card twice.

HTTPX has a simpler built-in option that retries only connection failures:

import httpx

transport = httpx.HTTPTransport(retries=3)
with httpx.Client(transport=transport, timeout=10) as client:
    response = client.get("https://httpbin.org/get")

retries here covers ConnectError and ConnectTimeout only, not 5xx responses. For status-based retries with HTTPX, write a small loop with backoff or use a library like Tenacity or Stamina.

Async Requests with HTTPX

This is the biggest reason to choose HTTPX. Requests is synchronous only; HTTPX has an AsyncClient with the same API, so you can make many requests concurrently with asyncio:

# async_fetch.py
import asyncio
import time

import httpx


async def main() -> None:
    async with httpx.AsyncClient(base_url="https://httpbin.org", timeout=10.0) as client:
        start = time.perf_counter()
        responses = await asyncio.gather(*(client.get("/delay/1") for _ in range(5)))
        elapsed = time.perf_counter() - start
        print([r.status_code for r in responses], f"{elapsed:.1f}s")


asyncio.run(main())

[200, 200, 200, 200, 200] 2.5s

Each /delay/1 call takes at least a second on the server. Sequentially, five would take more than five seconds; concurrently they overlap, and the whole batch finishes in about the time of the slowest one plus network overhead (your timing will vary). Every method is the same as the sync client, just awaited.

Two practical notes:

If you fire off hundreds of requests, limit concurrency with an asyncio.Semaphore or the client's limits=httpx.Limits(max_connections=...), both to protect the server and to avoid exhausting your own connections.
Async only helps inside async code. In a FastAPI app or any asyncio program, use AsyncClient. In a regular script, a thread pool with a sync client is also a perfectly good way to parallelize.

HTTP/2

With the http2 extra installed, HTTPX can negotiate HTTP/2:

import httpx

with httpx.Client(http2=True) as client:
    response = client.get("https://www.google.com/")
    print(response.http_version)

HTTP/2

HTTP/2 multiplexes many requests over one connection, which mainly helps when you make lots of concurrent requests to the same host. For a handful of sequential calls you won't notice a difference. Requests only supports HTTP/1.1.

Streaming Large Downloads

By default, both libraries read the whole response body into memory. For large files, stream it in chunks instead:

# download_httpx.py
import httpx

with httpx.stream("GET", "https://httpbin.org/bytes/50000") as response:
    response.raise_for_status()
    with open("download.bin", "wb") as f:
        for chunk in response.iter_bytes(chunk_size=8192):
            f.write(chunk)

# download_requests.py
import requests

with requests.get("https://httpbin.org/bytes/50000", stream=True, timeout=10) as response:
    response.raise_for_status()
    with open("download.bin", "wb") as f:
        for chunk in response.iter_content(chunk_size=8192):
            f.write(chunk)

Memory use stays at roughly one chunk no matter how large the file is. The with block makes sure the connection is released even if you stop reading early.

Logging and Hooks

HTTPX supports event hooks that run on every request and response, which is a clean place for logging or metrics:

import httpx


def log_request(request: httpx.Request) -> None:
    print(f"-> {request.method} {request.url}")


def log_response(response: httpx.Response) -> None:
    print(f"<- {response.status_code} {response.request.url}")


with httpx.Client(event_hooks={"request": [log_request], "response": [log_response]}) as client:
    client.get("https://httpbin.org/get")

-> GET https://httpbin.org/get
<- 200 https://httpbin.org/get

A response hook that calls response.raise_for_status() is a neat way to make every request on a client raise on error statuses. Requests has a similar hooks={"response": [...]} argument.

Testing Code That Makes HTTP Calls

You don't want unit tests hitting real APIs. HTTPX ships with MockTransport, which routes requests to a function you write:

# test_mock_transport.py
import httpx


def handler(request: httpx.Request) -> httpx.Response:
    if request.url.path == "/users/1":
        return httpx.Response(200, json={"id": 1, "name": "Ada"})
    return httpx.Response(404, json={"detail": "not found"})


client = httpx.Client(transport=httpx.MockTransport(handler), base_url="https://api.example.com")
print(client.get("/users/1").json(), client.get("/users/2").status_code)

{'id': 1, 'name': 'Ada'} 404

If your code accepts a client as a parameter instead of creating one internally, tests can pass in a mocked client like this with no patching at all. For Requests, the responses and requests-mock libraries fill the same role, and respx offers a richer mocking API for HTTPX.

Requests vs HTTPX at a Glance

	Requests	HTTPX
Sync API	Yes	Yes, nearly identical
Async API	No	`AsyncClient`
Default timeout	None (waits forever)	5 seconds
Follows redirects	Yes, by default	Only with `follow_redirects=True`
HTTP/2	No	Optional extra
Built-in retries	Via urllib3 `Retry` (status and connection)	Connection failures only
Mocking	Third-party (`responses`)	Built-in `MockTransport`
Maturity and ecosystem	Huge, very stable	Mature, widely used

Which One Should You Use?

Use Requests for scripts, CLIs, and sync codebases that already depend on it. It's stable, documented everywhere, and its urllib3-based retries are excellent. Just never forget timeout.
Use HTTPX for new projects, anything async (FastAPI, asyncio workers), when you want HTTP/2, or when you value safer defaults and built-in test transports. Many modern SDKs, including several official API clients, are built on HTTPX.

Because the APIs are so close, switching later is cheap. The main things to watch when porting are redirects, timeouts, and exception names.

If you're pulling data out of HTML pages rather than calling an API, see web scraping with Requests and Beautiful Soup.

Conclusion

The fundamentals are the same in both libraries: pass params instead of building URLs, use json= for JSON bodies, always have a timeout, call raise_for_status() and handle transport errors separately, and reuse a session or client for repeated calls. Layer on retries for transient failures and stream big downloads.

Requests remains a solid choice for straightforward synchronous code. HTTPX gives you the same feel with async support, HTTP/2, and stricter defaults, which makes it the better starting point for most new projects.