Alerts that carry a runbook link are acknowledged faster and silenced less often than alerts without one

Hypothesis

Prometheus alerting rules, as the documentation describes, take an annotations clause for longer information such as alert descriptions or runbook links, and other alerting systems commonly offer an equivalent field. Whether a rule actually has a runbook link, and whether the link leads to a current page, varies from rule to rule within the same team. The hypothesis is that the presence of a working runbook link changes on-call behaviour: pages from such rules are acknowledged sooner, resolved sooner, and are less often answered by silencing or muting the rule, because the responder has a first action instead of a search. A second, weaker hypothesis is that rules without runbooks accumulate more silences over time and are more likely to be deleted without replacement.

Prediction

Within one team's paging history, after grouping by rule, rules with a working runbook link show a shorter median time from page to acknowledgement and from page to resolution, and a lower share of pages followed by a silence of the same rule within an hour, than rules without a link or with a broken one. The difference is expected to be larger at night and for responders who have not owned the service long, and near zero for the few rules that fire daily and are known by heart.

Proposed test

1. Export, for a period of several months, every page with its rule, timestamps for firing, acknowledgement and resolution, and the responder; export silences with their creator, time and matched rule.
2. For each rule, record at the start and end of the period whether it had a runbook annotation and whether the link resolved to a page that had been edited within the last year.
3. Compare the timing and silence measures between rules with and without working runbooks, per responder and per time of day, reporting distributions rather than means.
4. Where a team adds runbooks to previously bare rules during the period, compare the same rules before and after, which controls for the rule's nature better than a cross-rule comparison.

Status

No result is claimed. The obvious confounder is that well-understood, frequently firing rules are the ones that get runbooks written, so the cross-rule comparison in step 3 may show a difference that has nothing to do with the link; the within-rule comparison in step 4 is the part of the test that matters.