Accounting for selected, skipped, and executed checks in a validation result

Goal

Prevent a passing command from being reported as successful coverage when the intended checks were skipped, deselected, or never discovered.

Prerequisites

Know which behaviors the validation is meant to exercise and how the project reports discovery, execution, and skip conditions. Use the project’s own check entry points where available.

Steps

1. State the intended check set before running it. This can be a named suite, a behavioral subset, or a specific integration path; avoid an undefined claim that everything was tested.

2. Read the execution summary for discovered, selected, executed, and skipped work. Preserve collection errors and missing prerequisites as separate outcomes even when another part of the command succeeds.

3. Compare the actual selected set with the intended one. Investigate path filters, markers, naming conventions, and configuration when expected checks are missing; do not compensate by reporting only the command’s exit status.

4. For skipped checks, record the reason and whether it blocks the requested conclusion. Obtain the authorized prerequisite or choose a valid alternative observation; do not remove protective conditions merely to make a check run.

5. Validate the reporting path with a fixture in which every relevant check is skipped and another in which the filter selects nothing. The final summary should state that the intended behavior remains unverified.

Expected result

A validation report distinguishes passed executions from unexercised behavior. It tells the reviewer whether the observed command result supports the acceptance claim and what prerequisite would close any remaining gap.

Limits and test basis

This is an original accounting protocol, not a statement about universal test-runner exit codes. Different tools treat empty or skipped suites differently. No runner behavior was tested here, and the applicable project configuration remains authoritative.