What clustering does

Clustering groups points such that:

• points within a cluster are similar
• points across clusters are less similar

Unlike classification, clustering gets no labels at all. Picture the iris dataset with its species labels stripped away: you can still spot the lower-left group of flowers with your own eyes, but you no longer have a name for it, and it’s not obvious that the remaining cluster is actually two overlapping species. That’s the whole game — find the groups and decide how many there really are.

  flowchart LR
  X[Data points] --> C[Clustering algorithm] --> G["Cluster labels (groups)"]

Why clustering shows up everywhere

Hands-On Machine Learning lists clustering as useful far beyond “finding groups”:

• customer segmentation — group users by purchase/activity patterns to target marketing
• data analysis — cluster first, then study each group separately
• dimensionality reduction — replace a feature vector with its k distances to each cluster centroid (a k-dimensional “affinity” vector)
• anomaly detection — a point with low affinity to every cluster is suspicious
• semi-supervised learning — label one representative per cluster, then propagate that label to the rest
• image segmentation — cluster pixels by color to simplify an image

Similarity and distance

Most clustering methods rely on a notion of similarity, often distance.

Common distances:

• Euclidean (geometry)
• Manhattan
• cosine distance (common for text/embeddings)

Important: scaling impacts clustering

If your features are on different scales, distance-based clustering can fail.

Use scaling (StandardScaler/MinMaxScaler) when appropriate.

Your first clustering

import numpy as np
from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
 
X, y_true = make_blobs(n_samples=300, centers=4, cluster_std=0.8, random_state=42)
 
kmeans = KMeans(n_clusters=4, n_init=10, random_state=42)
labels = kmeans.fit_predict(X)
 
print("Cluster sizes:", np.bincount(labels))
print("Silhouette score:", round(silhouette_score(X, labels), 3))

import numpy as np
from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
 
X, y_true = make_blobs(n_samples=300, centers=4, cluster_std=0.8, random_state=42)
 
kmeans = KMeans(n_clusters=4, n_init=10, random_state=42)
labels = kmeans.fit_predict(X)
 
print("Cluster sizes:", np.bincount(labels))
print("Silhouette score:", round(silhouette_score(X, labels), 3))

Clustering as dimensionality reduction

One application from the list above is worth seeing in code: once you have kk centroids, you can describe every point by its distance to each centroid instead of its original features. That turns an arbitrarily large feature vector into a kk-length affinity vector — a form of dimensionality reduction:

from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
 
X, _ = make_blobs(n_samples=100, centers=5, random_state=42)
 
kmeans = KMeans(n_clusters=5, n_init=10, random_state=42).fit(X)
X_dist = kmeans.transform(X)  # distance to each of the 5 centroids
 
print("Original shape:", X.shape)
print("Affinity-vector shape:", X_dist.shape)

from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
 
X, _ = make_blobs(n_samples=100, centers=5, random_state=42)
 
kmeans = KMeans(n_clusters=5, n_init=10, random_state=42).fit(X)
X_dist = kmeans.transform(X)  # distance to each of the 5 centroids
 
print("Original shape:", X.shape)
print("Affinity-vector shape:", X_dist.shape)

Original shape: (100, 2)
Affinity-vector shape: (100, 5)

Original shape: (100, 2)
Affinity-vector shape: (100, 5)

kmeans.transform()kmeans.transform() is exactly this — every row becomes its kk distances to the centroids, instead of the raw feature values.

What makes clustering hard

There is usually no “ground truth”.

You validate using:

• domain sense (do clusters mean something?)
• internal metrics (silhouette score)
• stability across runs

Mini-checkpoint

If you’re clustering customers:

• what features would you use?
• what would a “useful” cluster look like in business terms?

Visualize it

Same points, two views. On the left every point stays unlabeled — just a cloud of grey dots, the way the data actually arrives. On the right, watch a clustering algorithm sweep through the points one by one, assigning each a group color, until crosshair centroids settle in and the picture reads “clustered.” Then it fades back to grey and sweeps again. Click to scatter a new dataset:

🧪 Try It Yourself

Exercise 1 – Generate Blobs and Cluster Them

Exercise 2 – Scale Before You Cluster

Exercise 3 – Score a Clustering with Silhouette

Next

Continue to K-Means Clustering Algorithm — the simplest and most widely used clustering algorithm, and the one every other technique gets compared against.