Picture a model that is right 95 times out of 100 and has never once caught the thing it was built to catch. It is easy to build. If one record in twenty is the kind you care about, a model that always answers “no” scores 95 percent accuracy without looking at a single input.
That model is useless as a product and very useful as a first step. Before anything clever, build something deliberately dull. It is not there to win. It is there so you can tell whether the clever model did.
A score means nothing on its own
Someone tells you their model scores 0.87. Is that good? There is no way to know. On a yes-or-no problem with an even split, it is respectable. On a problem where 90 percent of rows share one label, it is worse than guessing the majority every time.
A baseline turns a score into a comparison. It answers the question that matters most at the start: how much of this result comes from the model, and how much from the problem being easy? Without one, people argue about decimals that may not mean anything.
Three baselines worth an hour
- The majority class. Always predict the most common label (or the average, if you are predicting a number). It costs nothing and sets the floor. A model that cannot beat it has learned nothing, however long it trained.
- A logistic regression. Fast, cheap and easy to inspect. On tabular data it is harder to beat than its reputation suggests, and its coefficients show which inputs carry the signal before you hand the problem to something you cannot read.
- A rule. Whatever the person doing the job by hand would say: flag any invoice above a set amount from a supplier you have never paid before. A rule captures what the business already knows, and your users will hold the model to it anyway.
Reading the gap
The useful information is in the distances between the scores. If the logistic regression barely beats the majority class, the features probably do not carry much signal. A bigger model rarely fixes that. Better data, or a better-framed question, usually does.
If the sophisticated model barely beats the logistic regression, the extra complexity is not paying its way. It needs more compute, a heavier deployment and more explaining, for a gain your users may never notice. Ship the simple one.
If the sophisticated model beats everything by a clear margin, good. Now you have evidence, and a number to put in front of whoever asks why the new version costs more to run.
And if anything beats the baseline by an implausible margin, be suspicious before you are pleased. A near-perfect score on a hard problem usually means information about the answer has leaked into the inputs.
The comparison, in code
scikit-learn has a DummyClassifier for exactly this job. Here it is next to a logistic regression, on synthetic data where roughly one row in twenty is positive:
from sklearn.datasets import make_classification
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold, cross_validate
# Stand-in data: roughly 1 positive row in 20
X, y = make_classification(n_samples=2000, n_features=20,
weights=[0.95], random_state=0)
models = {
"always majority": DummyClassifier(strategy="most_frequent"),
"logistic": LogisticRegression(max_iter=1000),
}
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=0)
for name, model in models.items():
scores = cross_validate(model, X, y, cv=cv,
scoring=["accuracy", "balanced_accuracy", "f1"])
print(f"{name:16}",
f"accuracy {scores['test_accuracy'].mean():.3f}",
f"balanced {scores['test_balanced_accuracy'].mean():.3f}",
f"f1 {scores['test_f1'].mean():.3f}")StratifiedKFold keeps the rare class in every fold in the same proportion. max_iter is raised because the default can stop before the solver converges. And the loop scores three metrics, not one, for reasons the next section explains.
Some of the output you can predict without running it. The dummy’s accuracy will land close to 0.95, because that is the share of the majority class. Its balanced accuracy will be exactly 0.5 and its F1 will be zero, because it never predicts the positive class at all. Run it and see where the logistic regression lands against those three numbers.
The accuracy trap
Accuracy counts every correct answer equally. When one class dominates, the easy correct answers on the common class swamp the hard ones on the rare class, and the rare class is usually the reason the model exists.
Better choices, depending on what a mistake costs:
- Balanced accuracy averages the recall of each class, so the dummy scores 0.5 however lopsided the data is.
- Precision and recall for the class you care about, reported separately. Recall is how many of the real cases you caught. Precision is how many of your alarms were real.
- F1 combines the two into one number. That is convenient for ranking models, and it hides which of the two is weak.
Choose the metric before you train, based on what each kind of error costs. A missed case and a false alarm rarely cost the same. If a miss is expensive, recall leads and precision gets a minimum. If every alarm sends a person to check, precision leads. Deciding after seeing the results is how teams end up reporting whichever number looked best.
When the model struggles with the rare class, try class_weight="balanced" in the logistic regression before reaching for anything bigger. It tells the model that mistakes on the rare class cost more, and it is one argument, not a new architecture.
Keep the floor visible
Put the baseline at the top of every results table and leave it there for the life of the project. Every new model has to beat that line, on the metric you chose before training, by enough to pay for what it costs to build and run. Sometimes the boring model is the one you ship. Even when it is not, it is the one that tells you whether anything else deserves to be shipped.
