Imbalanced classification

class_weight and threshold tuning, resampling with imbalanced-learn inside a pipeline, and measuring success with precision-recall rather than accuracy.

Weights first

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import precision_recall_curve

clf = LogisticRegression(max_iter=1000, class_weight="balanced")
clf.fit(X_train, y_train)

# or give the classes explicit relative weights
clf2 = LogisticRegression(max_iter=1000,
                          class_weight={0: 1.0, 1: 8.0}).fit(X_train, y_train)

# tune the decision threshold rather than accepting 0.5
prob = clf.predict_proba(X_val)[:, 1]
prec, rec, thr = precision_recall_curve(y_val, prob)
print([(round(float(p), 2), round(float(r), 2)) for p, r in zip(prec, rec)][::40])

• class_weight="balanced" scales each class loss by the inverse of its frequency, so a rare positive counts as much as many negatives.
• Explicit weights let you encode the real cost ratio rather than the raw frequency ratio.
• Threshold tuning changes only the operating point, not the ranking; if ranking quality is poor, weights will not rescue it.
• Trees accept class_weight too; HistGradientBoostingClassifier exposes the equivalent through sample_weight at fit time.

Resampling inside the pipeline

from imblearn.over_sampling import SMOTE
from imblearn.pipeline import Pipeline as ImbPipeline
from imblearn.under_sampling import RandomUnderSampler
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler

pipeline = ImbPipeline([
    ("scale", StandardScaler()),
    ("over", SMOTE(random_state=42, k_neighbors=5)),
    ("under", RandomUnderSampler(random_state=42)),
    ("clf", LogisticRegression(max_iter=1000)),
])

# the samplers run only on the training portion of every fold
scores = cross_val_score(pipeline, X, y, cv=cv, scoring="average_precision")

• Use imblearn.pipeline.Pipeline, not the scikit-learn one: the scikit-learn pipeline would resample the validation fold as well.
• Requires imblearn to be installed separately; add it to your environment file so the pipeline stays loadable.
• SMOTE interpolates between a positive row and its nearest positive neighbours, so it suits continuous features and can create impossible rows on one-hot data.
• Combine over- and under-sampling only if each improves the validation metric on its own; otherwise you are just adding variance.

Measuring honestly

from sklearn.metrics import (average_precision_score, balanced_accuracy_score,
                             precision_score, recall_score)

threshold = 0.35        # chosen from the cost of each error, not from habit
y_pred = (prob > threshold).astype(int)

print("average precision", round(average_precision_score(y_test, prob), 4))
print("balanced accuracy", round(balanced_accuracy_score(y_test, y_pred), 4))
print("precision", round(precision_score(y_test, y_pred, zero_division=0), 4))
print("recall", round(recall_score(y_test, y_pred, zero_division=0), 4))

FAQ

Related

Splitting data and cross-validation strategies Linear models

Last refreshed 2026-09-18.