Clustering and manifold learning

KMeans and MiniBatchKMeans, DBSCAN and AgglomerativeClustering, silhouette scores, PCA for compression and t-SNE for exploration.

KMeans and its variants

from sklearn.cluster import KMeans, MiniBatchKMeans
from sklearn.metrics import silhouette_score
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

km = Pipeline([("scale", StandardScaler()),
               ("km", KMeans(n_clusters=5, n_init=10, random_state=42))])
labels = km.fit_predict(X)

print(np.bincount(labels))
print(round(silhouette_score(km[:-1].transform(X), labels), 3))

# for large data, MiniBatchKMeans trades a little quality for speed
mb = MiniBatchKMeans(n_clusters=5, batch_size=1024, n_init=10, random_state=42)
print(km.named_steps["km"].cluster_centers_.shape)   # (5, n_features)
print(km.named_steps["km"].inertia_)                 # within-cluster sum of squares

• n_init=10 runs the initialisation ten times and keeps the best inertia; the old default of a single run is why clusters sometimes looked arbitrary.
• Always scale first. KMeans minimises Euclidean distance, so the widest-range column decides the clusters.
• Set random_state for reproducibility, and use .predict(X_new) to assign new rows to the fitted centroids.

Density-based and hierarchical clustering

from sklearn.cluster import AgglomerativeClustering, DBSCAN
from sklearn.metrics import adjusted_rand_score

db = DBSCAN(eps=0.7, min_samples=10).fit(X_scaled)
print(np.unique(db.labels_, return_counts=True))   # -1 marks noise points

agg = AgglomerativeClustering(n_clusters=5, linkage="ward")
agg.fit(X_scaled)
print(adjusted_rand_score(db.labels_, agg.labels_))  # agreement between two views

PCA and t-SNE

from sklearn.decomposition import PCA
from sklearn.manifold import TSNE

pca = PCA(n_components=0.95, whiten=False, random_state=42).fit(X_train_scaled)
print(pca.n_components_, round(pca.explained_variance_ratio_.sum(), 3))

X_train_pca = pca.transform(X_train_scaled)
X_test_pca = pca.transform(X_test_scaled)      # reuse the fitted components

import matplotlib.pyplot as plt
plt.scatter(X_train_pca[:, 0], X_train_pca[:, 1], c=y_train, s=6, alpha=0.6)

tsne = TSNE(n_components=2, perplexity=30, init="pca",
            learning_rate="auto", random_state=42)
X_2d = tsne.fit_transform(PCA(n_components=50, random_state=42)
                          .fit_transform(X_train_scaled))

• PCA is a linear projection that maximises retained variance; it can be fitted inside a pipeline and applied to new rows.
• t-SNE preserves local neighbourhoods only. Distances between distant blobs, and empty space, mean nothing.
• Never feed t-SNE coordinates to a model as features — it has no transform method, so new rows cannot be placed consistently.
• If the first two PCA components carry a tiny share of the variance, a flat scatter plot is expected and not a bug.

FAQ

Related

Feature engineering and text features Missing data and outliers

Last refreshed 2026-09-18.