Establishing a baseline before training the first model

Goal

Know, in the project's own metric and on the project's own held-out data, what "no model" achieves, so that the value of every later model is measurable and a model that adds nothing is recognised before it is shipped.

Prerequisites

A fixed split (see training, validation and test sets), a chosen metric with its threshold or averaging rule written down, and access to whatever currently makes the decision: a rule, a lookup table, a human process or nothing.

Steps

1. Run a trivial predictor. scikit-learn's DummyClassifier and DummyRegressor ignore the features and predict, for example, the most frequent class, a class drawn from the empirical class prior, or the mean or median of the target; the API describes the classifier as serving as a simple baseline. Score it on the validation split with the real metric.
2. Write a one-line rule from domain knowledge (a threshold on one column, "same as last time", the most recent value). Score it the same way. Google's Rules of Machine Learning says not to fear launching without machine learning, that a heuristic gets part of the way, and that the first model should be simple while the infrastructure is got right.
3. If a process already exists, score its historical decisions on the same rows, so that the model is compared with what it would replace, not with zero.
4. Fit the simplest learning model appropriate to the data (a regularised linear or logistic model, a small tree) with default settings. Score it.
5. Record all four numbers with the split identifier, metric definition, date and code commit in the project notebook before touching a larger model.
6. Report every subsequent model as a delta against the strongest baseline, with an interval, and keep the baseline rows in every results table.

Expected result

A short table that answers "how much does learning add?" at any point in the project, and an early stop for problems where a rule is enough or where the data carries no signal.

Limits and test basis

The baseline says nothing about whether the metric is the right one; that decision precedes it. A strong rule can be a sign of leakage in the data rather than of a solved problem. This is a proposed protocol; the sources document the tools and the general advice, not results of following the protocol.