HTTP keep-alive and connection reuse: pools, idle timeouts and the stale-connection race

What it is

RFC 9112 states that HTTP/1.1 defaults to persistent connections, carrying multiple requests and responses over one connection. A connection stays open unless a message carries the close connection option or the peer speaks HTTP/1.0 without keep-alive; a client or server that does not support persistence must send close on every message. To remain persistent, every message needs a self-defined length (Content-Length or chunked encoding). Servers usually close inactive connections after a timeout whose length the specification does not constrain, connections can close at any time, and clients ought to limit the number of simultaneous connections to one server. MDN describes the same models and notes that pipelining, though defined, is not activated by default in modern browsers.

Why it matters

Each new connection costs a TCP handshake and, with TLS, a cryptographic handshake on top, plus a socket in TIME_WAIT afterwards. A client making many small requests to one host can spend more time establishing connections than transferring data, and a server can spend more CPU on TLS handshakes than on requests. Reuse removes that cost from every request after the first.

How to apply

• Keep one client object with a connection pool for the life of the process (requests.Session, httpx.Client, a Node Agent with keep-alive) instead of creating one per call. The Requests documentation states that keep-alive is automatic within a session.
• Consume or close every response body. The same documentation notes that connections return to the pool only once all body data has been read; an unread streamed response occupies a connection.
• Set the client's idle timeout below the server's and the proxy's. Otherwise the client picks a connection the server has just closed, and the first request after an idle period fails with a reset. RFC 9112 says implementations ought to anticipate such asynchronous closes and refers to RFC 9110 for when a request may be retried automatically; retry once, and only idempotent requests.
• Bound the pool per host to what the server accepts, and recycle connections after a number of requests or a maximum age so that long-lived connections do not pin a client to one backend behind a load balancer.
• Check response headers: a server that answers Connection: close defeats the pool silently.

Pitfalls

Middleboxes drop idle connections without telling either end; the open question on NAT idle timeouts covers this. A TLS connection authenticated with a client certificate is bound to that identity; do not share such a connection between tenants. HTTP/2 multiplexes many streams on one connection, so its pooling rules differ from HTTP/1.1.