Unsupervised learning
K-means, hierarchical and density-based clustering, PCA and manifold methods, and anomaly detection when nobody has labelled the odd cases.
Clustering
import numpy as np
from sklearn.cluster import DBSCAN, AgglomerativeClustering, KMeans
from sklearn.metrics import silhouette_score
from sklearn.preprocessing import StandardScaler
X_scaled = StandardScaler().fit_transform(X)
km = KMeans(n_clusters=6, n_init=10, random_state=42).fit(X_scaled)
labels = km.labels_
print(silhouette_score(X_scaled, labels)) # -1 to 1, higher is better
print(np.bincount(labels)) # tiny clusters are often noise
agg = AgglomerativeClustering(n_clusters=6, linkage="ward").fit(X_scaled)
db = DBSCAN(eps=0.6, min_samples=8).fit(X_scaled)
print((db.labels_ == -1).sum()) # -1 marks points as noise| Method | Finds | You must choose | Fails when |
|---|---|---|---|
| K-means | Roughly equal, spherical groups | k up front | Clusters differ wildly in size or shape |
| Hierarchical | A nested merge tree | Linkage and a cut height | Data is large; naive implementations are slow |
| DBSCAN | Dense regions plus outliers | eps and min_samples | Density varies across the dataset |
| HDBSCAN-style density methods | Clusters of varying density | Fewer knobs | Very high dimensions |
There is no ground-truth label to score against, so validate a clustering by whether the segments differ on something you did not cluster on — churn, revenue, support load. A stable silhouette with useless segments is a failed project.
Reducing dimensions
from sklearn.decomposition import PCA, TruncatedSVD
from sklearn.manifold import TSNE
pca = PCA(n_components=0.95, random_state=42).fit(X_scaled) # keep 95% of variance
print(pca.n_components_, pca.explained_variance_ratio_.sum())
X_pca = pca.transform(X_scaled)
# SVD works on sparse text matrices where PCA would densify them
svd = TruncatedSVD(n_components=100, random_state=42)
# t-SNE is for looking at data, not for feeding a model
emb = TSNE(n_components=2, perplexity=30, random_state=42).fit_transform(X_scaled)- Scale before PCA; the component directions follow whichever column has the largest variance.
- Always fit PCA on the training fold only. Fitting on everything, including the test set, is the same leak as scaling globally.
- t-SNE distances between distant clusters are not meaningful, and the picture changes with
perplexity. Use it to explore, never to select features.
Anomaly detection
from sklearn.ensemble import IsolationForest
from sklearn.neighbors import LocalOutlierFactor
iso = IsolationForest(contamination=0.02, random_state=42).fit(X_scaled)
print(iso.predict(X_scaled)[:10]) # -1 = flagged as anomalous
lof = LocalOutlierFactor(n_neighbors=20, contamination=0.02)
lof.fit_predict(X_scaled) # local density relative to neighbours💡
Every anomaly detector needs a review budget. Decide how many alerts per week a human will actually read, set
contamination to match that number, and treat the output as a ranked queue rather than a verdict.FAQ
How do I choose the number of clusters?
Plot the silhouette score and the within-cluster sum of squares across a range of k, but choose with the business in mind: a segment you cannot act on is not a segment. Two to eight groups is usually the useful range.
Can I use cluster labels as a feature?
You can, but fit the clusterer on training data only and reuse it to label the test and serving rows. Assigning clusters independently to each split produces inconsistent labels between train and production.
Related
Supervised algorithms in practice Interpretability, bias and fairness
Last refreshed 2026-09-18.