How far off were the variance assumptions behind sample-size calculations in small online experiments, and in which direction?

Open question

A sample-size calculation needs three inputs: the risk of a false positive, the risk of missing a real effect of a stated size, and the spread of the measurement. The NIST/SEMATECH handbook section on required sample sizes states the restriction plainly: the standard deviation must be known, and lacking an exact value one has to estimate it. In product experiments the estimate comes from last quarter's data, a similar metric, or a round guess, and it feeds a number of users per arm that decides how long the experiment runs. The wiki has a methodology for the calculation and an article on analysis pitfalls, but no evidence about how good the variance input tends to be.

The open question is therefore empirical: for experiments whose plan recorded the assumed standard deviation (or the assumed baseline rate for a proportion), what was the standard deviation actually observed in the experiment's data? Is the ratio of observed to assumed centred on one, or is it systematically above one because the estimate came from a calmer period, a cleaner segment or an aggregated metric with less spread than the per-user values? Does the error differ between conversion rates, where the variance follows from the rate, and continuous metrics such as revenue per user or session length, where heavy tails dominate? How often did the discrepancy leave the experiment underpowered for the effect size it was designed to detect, and what happened then: was it extended (and with what stopping rule), was it declared inconclusive, or was it read as if it had the planned power?

A secondary question: after several experiments, did teams adjust their planning inputs, for example by adding a margin to the assumed variance or by planning on the observed variance of a preceding A/A period?

What a useful answer contains

Per experiment: the metric type, the assumed variance or baseline rate and its origin, the planned sample size and effect size, the observed variance, the achieved sample size and the decision taken. Enough experiments from one team to see a pattern, with the period. The ratio of observed to assumed variance summarised as a distribution rather than a mean. Where an experiment was extended, the rule used and whether it was decided before the data was seen. Whether an A/A run or a pre-period was used to estimate variance, and how that estimate compared. Reports where the assumptions held are as valuable as reports where they failed.