Variance, standard deviation, MAD and IQR: reporting the spread

What it is

The NIST/SEMATECH handbook defines the common measures of scale: the variance (sum of squared deviations from the mean, divided by N−1), the standard deviation (its square root), the range (largest minus smallest), the average absolute deviation, the median absolute deviation (MAD, the median of the absolute deviations from the median) and the interquartile range (IQR, 75th minus 25th percentile). It notes that the variance is intended as an overall measure of spread but can be greatly affected by the tail behaviour, that the range uses only the two most extreme points, and that the IQR uses only the middle portion of the data. Its example with 10,000 random numbers makes the point: a normal sample has standard deviation 0.997 and MAD 0.681, a Cauchy sample has standard deviation 998.389 and MAD 1.16, and for the Cauchy distribution collecting more data does not provide a more accurate estimate of the mean or standard deviation. The Python statistics documentation separates pvariance/pstdev (population, divisor N) from variance/stdev (Bessel's correction, divisor N−1) and states that calling the population form on a sample gives the biased sample variance.

Why it matters

Two builds with the same mean latency but standard deviations of 5 ms and 50 ms are different products. A change smaller than the spread between runs is not a change. The measure of spread chosen decides whether tail events show up in the report at all.

How to apply

• Attach a spread and a count to every location: "median 120 ms, IQR 95–160 ms, n = 4,812" or "mean 3.2 s, SD 0.4 s, n = 30".
• Use the standard deviation for roughly symmetric, light-tailed data; for skewed or heavy-tailed data report the IQR or MAD, plus a high percentile when the tail matters.
• Use the N−1 form when the data are a sample of what could have happened (almost always); use the population form only when the data are the whole population of interest.
• Keep the standard deviation (spread of the data) apart from the standard error (standard deviation divided by the square root of n, the uncertainty of the mean). Label which one an error bar shows.
• Print standard deviations, not variances: variance carries squared units and no intuition.

Pitfalls

A bare "±" that could mean standard deviation, standard error or an interval. A standard deviation on proportions near 0 or 1, where the spread is asymmetric and bounded. Spread computed from pre-aggregated daily averages, which hides the within-day variation. Outliers removed before computing the spread without saying so.