Clustering

Clustering groups unlabeled data by similarity. This landing page explains the main definitions of a cluster, helps choose an algorithm, and links to detailed pages on K-Means, DBSCAN and hierarchical clustering.

1. What Clustering Does

Clustering assigns unlabeled points to groups so that points in a group are more similar to each other than to points in other groups. "Similar" is defined by a distance (Euclidean, cosine, Jaccard) and by the algorithm's notion of a cluster: closeness to a center, membership in a dense region, or position in a merge tree. Typical uses are customer segmentation, grouping documents or embeddings into topics, image color quantization, deduplication, and building features for downstream supervised models.

2. Algorithms

3. Evaluating Clusters

Without labels, use internal metrics: silhouette (cohesion vs separation, -1 to 1), Davies–Bouldin (lower is better), Calinski–Harabasz (higher is better). They favor convex clusters, so they can mislead for density methods. If some labels exist, external metrics such as Adjusted Rand Index and Normalized Mutual Information compare clusterings to them. Always profile clusters (feature means, examples) and check stability across seeds.

from sklearn.cluster import KMeans, AgglomerativeClustering, HDBSCAN
from sklearn.datasets import make_blobs
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import silhouette_score, davies_bouldin_score

X, _ = make_blobs(n_samples=1500, centers=5, random_state=42)
X = StandardScaler().fit_transform(X)
for name, m in [("kmeans", KMeans(5, n_init=10, random_state=0)),
                ("ward",   AgglomerativeClustering(5)),
                ("hdbscan", HDBSCAN(min_cluster_size=20))]:
    lab = m.fit_predict(X)
    mask = lab != -1                      # ignore HDBSCAN noise for the metrics
    print(name, round(silhouette_score(X[mask], lab[mask]), 3),
          round(davies_bouldin_score(X[mask], lab[mask]), 3))

4. Pitfalls

• Clustering unscaled features, or features dominated by a few high-variance columns.
• Clustering high-dimensional raw data; reduce first with dimensionality reduction.
• Reading cluster structure off a t-SNE plot and then clustering the 2-D projection instead of the data.
• Treating any clustering as "the" answer: clustering always returns groups, even for uniform noise.

5. A Practical Workflow

Most clustering projects follow the same pipeline. Feature choice matters more than the algorithm: the clusters can only reflect the variables you give them, so decide up front what "similar" should mean for the business question (similar spending patterns, similar text meaning, similar sensor behavior). Scaling comes next, because Euclidean distance lets a feature measured in thousands drown out one measured in fractions; standardize numeric columns and encode categoricals deliberately (one-hot columns inflate distances, Gower distance is an alternative for mixed data).

For more than a few dozen dimensions, reduce before clustering. Distances concentrate in high dimensions, meaning the nearest and farthest neighbors end up at similar distances and every algorithm struggles. PCA to a few dozen components is a safe default; UMAP to 5–15 dimensions often works well before HDBSCAN on embeddings. Then fit and tune (k for K-Means, min_cluster_size for HDBSCAN, the cut height for hierarchical), validate with metrics and stability checks, and finally profile: compute per-cluster feature means, read sample members, and give each cluster a human name. A cluster nobody can describe is rarely useful.

6. Choosing the Number of Clusters

For algorithms that need k, there is no single correct method; combine several signals:

• Elbow plot: plot within-cluster sum of squares (inertia) against k and look for the point where the decrease flattens. It always decreases, so the elbow is often ambiguous.
• Silhouette: pick the k with the highest mean silhouette, and inspect per-cluster silhouette plots for clusters that are mostly negative.
• Gap statistic (Tibshirani, Walther & Hastie, 2001): compares inertia to that of uniformly distributed reference data, which also tests whether there is any structure at all.
• Information criteria: for Gaussian mixtures, BIC penalizes extra components and gives a principled choice.
• Use-driven: if marketing can only run four campaigns, k=4 may be the right answer regardless of the metrics.

Density-based methods sidestep k but move the decision to density parameters, so the judgment does not disappear; it changes form.

7. Real-World Uses

In production, the hard part is usually stability over time: refitting next month renumbers clusters and can split or merge segments. Match new clusters to old ones by centroid distance (Hungarian matching), or freeze centroids and only assign new points, refitting on a schedule with a review.

8. Further Reading

• MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations (K-Means).
• Ester, M., Kriegel, H.-P., Sander, J. & Xu, X. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise (DBSCAN). KDD.
• Campello, R., Moulavi, D. & Sander, J. (2013). Density-based clustering based on hierarchical density estimates (HDBSCAN).
• Rousseeuw, P. (1987). Silhouettes: a graphical aid to the interpretation and validation of cluster analysis.
• scikit-learn User Guide, "Clustering" chapter.

9. Related Topics