Running mutation testing without drowning in survivors

Goal

Use mutation testing as a routine check that tests detect faults, without turning it into a CI bottleneck or a report full of survivors nobody acts on. Whether the score predicts escaped defects is a separate hypothesis entry; this is the operating procedure.

Prerequisites

A deterministic, order-independent unit test suite; a mutation tool for the language (PIT for the JVM, StrykerJS for JavaScript and TypeScript, mutmut for Python, cargo-mutants for Rust); line coverage already measured, since uncovered code produces only "no coverage" mutants.

Steps

1. Run the tool on one module with logic in it, not on the whole codebase, and read the report. PIT's documentation lists the outcomes per mutant: killed, survived, no coverage, timed out, plus non-viable and error states. Stryker's metrics page defines the mutation score as detected divided by valid mutants, where detected counts kills and timeouts.
2. Open each survivor in the code. It is one of three things: a missing assertion (the test ran the line but checked nothing that depends on it), a missing case (a boundary no test reaches), or an equivalent mutant that cannot change observable behaviour, which PIT's documentation illustrates with >= 1 versus > 1 when the value is always 2.
3. Fix the first two kinds by strengthening tests. For equivalents, exclude the mutator or annotate the line as the tool allows instead of writing contrived tests.
4. Bound runtime: scope the run to changed files or use incremental mode (StrykerJS --incremental reuses results for mutants whose covering tests did not change); set a per-mutant timeout; switch off mutators that only generate noise for the codebase (PIT skips lines with calls to common logging frameworks by default).
5. Put the bounded run in CI on the diff and a full run on a schedule with the report published, not gating.
6. Track the score per module and use it as a ratchet ("no lower than the last run") rather than a fixed global target.

Expected result

Each change yields a short list of concrete missing assertions before review, and the score of the modules that matter rises slowly instead of being argued about.

Limits and test basis

Runtime multiplies with the number of mutants, and flaky or order-dependent tests make results meaningless. Scores are not comparable across tools or mutator sets. A high score on trivial code says little; effort belongs on modules with branching logic. No measurement is claimed.