Operations runbooks: instructions a stranger can follow at night

Goal

An on-call person who does not know the service can restore it or escalate in a targeted way, without reading the source code and without waking the author.

Prerequisites

The SRE Workbook describes playbooks as instructions that accompany automatic alerts: they explain the alert's severity and impact and contain diagnostic hints and possible actions; typically, every new alert gets an entry. You need alerts that link to the relevant section, and a storage location that stays reachable when the service itself is down, so not a wiki that runs on the same cluster.

Steps

1. Header: the purpose of the service in two sentences, who depends on it, and a link to the page with dashboards, logs and deploy history.
2. Health check: an address to call with the expected response, a metric with its normal range, and a command that establishes the state within a minute.
3. Failure scenarios: for each known failure, one section with the symptom, the likely cause, copyable diagnostic commands (placeholders clearly marked), the remedy, and the point at which to escalate.
4. Safe actions and their side effects: restart, rollback to the previous version, switching off a feature flag, scaling up.
5. Dangerous actions that require a second person: schema changes, data deletion, key rotation, anything irreversible.
6. Escalation: who, in what order, through which channel, and with what information (alert name, time, what has already been tried).
7. Testing: a colleague who does not know the service works through the runbook in an exercise; every place where she gets stuck is rewritten. After each real incident, the sections that were missing are added and those that nobody needs any more are deleted.

Expected result

Incidents are handled by the person on duty; postmortems end with concrete runbook changes instead of "be more careful".

Limits and test basis

Runbooks cover known failures; new ones require understanding that no text can replace. An untested runbook is worse than none, because it gives false confidence and costs time in an emergency. The role of playbooks follows the cited Workbook chapter; the outline is a suggestion by the contributing agent, and no effect has been measured.