Timeouts, retries and backoff with jitter

Goal

Make a client resilient to transient failures without amplifying an outage or duplicating work.

Prerequisites

Knowledge of which operations are idempotent (or protected by idempotency keys) and what the server signals on overload (429 or 503 with Retry-After).

Steps

1. Set a connect timeout and a read timeout on every remote call; choose them from the caller's own deadline, not from the slowest observed case.
2. Retry only on transient outcomes: connection errors, timeouts, 429, 503, 502/504 from intermediaries. Never retry on 4xx validation errors.
3. Retry only idempotent operations, or operations with an idempotency key; a plain POST is retried only if the server documents idempotent behaviour.
4. Space retries exponentially (base × 2^attempt) with random jitter, as the cited AWS analysis recommends, and cap both the delay and the number of attempts.
5. Honour Retry-After when the server sends it; it overrides the computed delay.
6. Add a circuit breaker or budget so that a failing dependency does not consume all client capacity.

Expected result

Clients recover from short outages automatically, load on a recovering server ramps up smoothly, and no operation is executed twice by accident.

Limits and test basis

Retries add latency; interactive paths may prefer failing fast. Backoff parameters need tuning per dependency. The recommendations follow the cited sources and the practice of this wiki's example client.

Advisory rate-limit headers

Some servers announce their limits before any 429. An IETF draft (draft-ietf-httpapi-ratelimit-headers) defines a RateLimit field that carries the remaining quota and the seconds until it resets, and a RateLimit-Policy field that describes the policy; other servers send vendor-specific headers with the same meaning. When such fields are present, read them and slow down before the quota reaches zero rather than after; treat them as hints that may be absent or change shape, and keep the backoff of steps 4 and 5 as the fallback. Never let an advisory header raise the request rate above what the caller's own budget allows.