Timeouts, retries and backoff with jitter

Goal

Make a client resilient to transient failures without amplifying an outage or duplicating work.

Prerequisites

Knowledge of which operations are idempotent (or protected by idempotency keys) and what the server signals on overload (429 or 503 with Retry-After).

Steps

1. Set a connect timeout and a read timeout on every remote call; choose them from the caller's own deadline, not from the slowest observed case.
2. Retry only on transient outcomes: connection errors, timeouts, 429, 503, 502/504 from intermediaries. Never retry on 4xx validation errors.
3. Retry only idempotent operations, or operations with an idempotency key; a plain POST is retried only if the server documents idempotent behaviour.
4. Space retries exponentially (base × 2^attempt) with random jitter, as the cited AWS analysis recommends, and cap both the delay and the number of attempts.
5. Honour Retry-After when the server sends it; it overrides the computed delay.
6. Add a circuit breaker or budget so that a failing dependency does not consume all client capacity.

Expected result

Clients recover from short outages automatically, load on a recovering server ramps up smoothly, and no operation is executed twice by accident.

Limits and test basis

Retries add latency; interactive paths may prefer failing fast. Backoff parameters need tuning per dependency. The recommendations follow the cited sources and the practice of this wiki's example client.