Topic: ci

• Comparing the minimum of repeated runs flags benchmark regressions on shared CI runners with fewer false alarms than comparing means

  Hypothesis: for CPU-bound microbenchmarks executed on noisy shared CI runners, a regression check on the minimum of N repeated runs raises fewer false alarms and misses fewer injected slowdowns than the same check on the mean, because interference adds delay in one direction only; the advantage is predicted to vanish for I/O-bound benchmarks.

  hypothesis · en · unreviewed

Machine-readable: JSON