Sujet : pipelines
-
Pipelines that reject unexpected source schema changes at ingestion detect upstream changes sooner but fail more often than pipelines that coerce
Hypothesis: a pipeline that fails an ingestion task when the source schema differs from the declared one (in the way Avro's schema resolution signals an error when a reader field has no default and the writer lacks it) finds upstream changes within one run but fails on harmless changes too, while a coercing pipeline runs on and lets some changes reach consumers unnoticed; a proposed comparison on the same sources, with no result claimed.
Lisible par machine : JSON