Freshness and row-count checks on raw source tables catch most pipeline incidents earlier than column-level tests downstream

Hypothesis

Warehouse incidents (a report shows wrong numbers, a downstream job fails) have their root cause upstream more often than in the model that fails. If that is so, then two checks that need no knowledge of the data's meaning, source freshness (was the raw table loaded within its expected cadence, as the cited dbt configuration computes from a loaded_at_field) and interval volume (did the load deliver a plausible row count), will fire earlier and for a larger share of incidents than the column-level data tests the cited documentation describes (not-null, unique, accepted values, relationships) placed on transformed models. The hypothesis does not claim that column-level tests are unnecessary; it claims that, ordered by detection time, the source-layer checks come first for most incidents and that a team with limited effort should implement them before anything else.

Prediction

For a warehouse that records incidents with a root cause and the first failing check, at least half of the incidents whose root cause is an upstream outage, a partial extract or a scheduling failure will be detected by a freshness or volume check before any column-level test fails, and the median time between the first source-layer alert and the first downstream test failure will be at least one scheduling interval. Incidents caused by logic errors in transformations will show the opposite order, and their share decides whether the hypothesis holds overall.

Proposed test

1. Enable freshness and volume checks on every raw source and keep the existing downstream tests unchanged; record every check result with a timestamp.
2. For a period of several months, record every incident with its root cause category (upstream, extraction, scheduling, transformation logic, schema change) and the timestamps of the first firing check of each kind.
3. Count, per category, which check kind fired first and the lag between them; report the proportions with their uncertainty and the number of incidents no check caught.
4. Pre-register the threshold (half of incidents) and the categories before collecting data.

Status

No result is claimed. Confounders: teams that already have mature downstream tests will have fewer transformation incidents; volume thresholds set too wide will miss partial loads and bias the count against the hypothesis; incidents that nobody records cannot be counted.