Designing an operations dashboard: one question per panel, one screen per audience

Goal

Build a dashboard that a person paged at night can read in under a minute and that answers a fixed set of questions, rather than a wall of every metric the service exports.

Prerequisites

A metrics source with consistent names and labels, the list of alerts that will link to the dashboard, and a named audience (on-call responder, service owner, capacity planner). Grafana's best-practice guide states the rule this method follows: a dashboard should tell a story or answer a question, and if it has no goal it may not be needed. The SRE book says the same of dashboards in general: they should answer basic questions about the service.

Steps

1. Write down the questions, in the order a responder asks them: Is the service meeting its objective right now? Is the problem in traffic, errors or latency? Which dependency or instance is involved? What changed recently?
2. Assign exactly one panel per question and title the panel with the question's subject ("Error ratio, last 30 min", not "http_requests_total"). Remove any panel without a question.
3. Order top to bottom from general to specific, as the Grafana guide suggests: objective and user-facing signals first, per-dependency and per-instance rows below, resource usage last.
4. Normalise: same time range on every panel, percentages instead of raw counts where machines differ in size, base units with unit-aware axes, thresholds coloured by meaning.
5. Replace copies with template variables for environment, cluster and instance so one dashboard serves all of them; the guide names this as the way to prevent sprawl.
6. Add a deployment or change marker so "what changed" is visible without leaving the page.
7. Link each alert to this dashboard with the variables pre-filled, and link panels to the drill-down dashboard or trace search.
8. Store the dashboard JSON in version control and review it after each incident: add a panel only for a question that was actually asked, delete panels nobody used.

Expected result

One screen per audience with a handful of panels, each answering one question, reachable from the alert that made the responder open it.

Limits and test basis

This is an authoring protocol, not a measured comparison; whether fewer panels shorten diagnosis is stated separately as a hypothesis. Dashboards cannot replace alerts (nobody watches them continuously) and show aggregates that can hide a single bad instance unless a per-instance panel exists.