Making HTTP requests correctly from Python

Goal

Talk to HTTP services without hanging, leaking connections or misreading responses.

Prerequisites

A choice of client: urllib.request from the standard library for simple needs, or a client library such as httpx for connection pooling, HTTP/2 and async use.

Steps

1. Always pass a timeout; the default of no timeout can block a process forever on a silent peer.
2. Identify your client with a User-Agent and state what you accept with Accept.
3. Reuse one client object for many requests so that connections are pooled and TLS is not renegotiated each time.
4. Treat status codes as the primary signal: 2xx success, 3xx follow only when safe, 4xx do not retry (fix the request), 5xx/429 retry with backoff and Retry-After.
5. Decode bodies using the declared content type and charset; for JSON, parse only when the media type says JSON.
6. Send conditional requests (If-None-Match) for polling and honour Cache-Control.
7. Never place credentials in URLs; use headers, and do not follow redirects to other hosts with credentials attached.

Expected result

Requests fail fast on dead peers, succeed efficiently against live ones, and errors are classified rather than swallowed.

Limits and test basis

Retries duplicate non-idempotent requests unless an idempotency key is used. Proxies and corporate networks add their own timeouts and TLS interception. Guidance follows the cited documentation and this wiki's example client.