HowToCodePage

Estimators and fit/predict

The uniform API behind every scikit-learn model: fit, predict, transform, and the data shapes the library silently expects.

The estimator API

import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

X = np.random.default_rng(0).normal(size=(200, 4))     # (n_samples, n_features)
y = (X[:, 0] + X[:, 1] > 0).astype(int)                # (n_samples,)

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y)

clf = LogisticRegression(max_iter=1000)     # 1. construct with hyperparameters
clf.fit(X_train, y_train)                   # 2. learn from data
preds = clf.predict(X_test)                 # 3. labels
probs = clf.predict_proba(X_test)[:, 1]     # 4. probabilities
clf.score(X_test, y_test)                   # mean accuracy

clf.coef_.shape          # (1, 4) — what was learned
clf.classes_             # array([0, 1]) — label order matches predict_proba columns
MethodOn estimatorsPurpose
fit(X, y)AllLearn parameters; returns self
predict(X)SupervisedOutput a label or value
predict_proba(X)Classifiers that support itClass probabilities
transform(X)TransformersProduce a new feature matrix
fit_transform(X)TransformersBoth, often more efficient together
get_params() / set_params()AllRead and change hyperparameters; used by grid search
⚠️
Never call fit (or fit_transform) on the full dataset before splitting. Any statistic learned from the whole set — scaler mean, imputer median, target encoding, feature selection — has already seen the test rows, and your reported score is optimistic. Split first, then fit inside the training fold.

Shapes and preprocessing basics

from sklearn.preprocessing import StandardScaler, OneHotEncoder

# X must always be 2-D, even for a single feature
X = X.reshape(-1, 1)          # (n,) -> (n, 1), NOT (1, n)

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)   # fit on train
X_test_scaled = scaler.transform(X_test)         # reuse the same mean/std

scaler.mean_, scaler.scale_          # the learned statistics
scaler.transform([[0.0, 0.0, 0.0, 0.0]])   # 1-D input also accepted

ohe = OneHotEncoder(handle_unknown="ignore", sparse_output=False)
ohe.fit([["red"], ["blue"], ["green"]])
ohe.transform([["blue"], ["teal"]])   # unseen category -> all zeros, no crash
  • Shapes are (n_samples, n_features). A 1-D array passed to fit usually raises, but a wrongly transposed one may not — check X.shape before you blame the model.
  • Pandas DataFrames work if all columns are numeric; a stray object column causes a conversion error late in the pipeline.
  • fit_transform on training data, plain transform on anything else. Calling fit_transform on the test set is the same leak in disguise.
  • scale_ and mean_ end with an underscore by convention: attributes with a trailing underscore only exist after fit.

Choosing a first model

ProblemStart withWhy
Binary / multiclass classificationLogisticRegressionFast, interpretable coefficients, strong baseline
Non-linear tabular classificationHistGradientBoostingClassifierHandles mixed types and missing values well
RegressionRidgeRegularised linear baseline
Small data, few featuresKNeighborsClassifierNo training phase, easy to reason about
Unknown structureKMeans, PCAClustering and dimensionality reduction
Imbalanced classesclass_weight="balanced"Reweights without resampling
from sklearn.dummy import DummyClassifier

# always beat a trivial baseline first
dummy = DummyClassifier(strategy="most_frequent").fit(X_train, y_train)
print("baseline", dummy.score(X_test, y_test))

# and check the data before tuning anything
print(np.bincount(y))            # class balance
print(np.isnan(X).sum())         # missing values per feature

FAQ

Why does predict_proba return two columns?
One column per class, ordered as classes_. For binary problems column 0 is the negative class and column 1 the positive one, so take [:, 1] when you want the probability of the positive label.
What does random_state do?
It seeds every randomised step, making splits and models reproducible. Set it while developing and evaluating so a changed score means a changed model, not a changed split.

Pipelines and preprocessing Machine learning in one page

Last refreshed 2026-09-18.