Does K-means maximise the distance between clusters?

Not directly, and the distinction matters when you are debugging a bad result. The objective K-means actually minimises is the within-cluster sum of squared distances, meaning how tightly each group hugs its own centre. Clusters usually end up well separated as a side effect, but nothing in the algorithm penalises two centroids sitting close together. That is why K-means can happily split one real group into two when you ask for too many clusters.

When should I use K-medoids instead of K-means?

When outliers are a problem. A K-means centroid is an average, so a single extreme point drags it away from where the actual cluster sits. A K-medoid is a real data point chosen from the cluster, which cannot be pulled somewhere no data exists. The cost is speed: K-medoids is considerably more expensive to compute, so it suits smaller datasets or ones where you know the data is messy.

What does fuzzy c-means give me that K-means doesn't?

Degrees of membership rather than a hard assignment. K-means says a customer is in segment 3; fuzzy c-means says they are 60% segment 3 and 40% segment 1. That is more honest when the underlying groups genuinely overlap, which is most real customer data, and it gives you a confidence signal: points with membership split evenly across clusters are the ones sitting on a boundary and worth looking at individually.

Why is centroid-based clustering bad at non-spherical shapes?

Because assignment is by distance to a centre, which implicitly assumes clusters are roughly round blobs of similar size. Give it two interleaved crescents and it will slice straight through both, because points at the tip of one crescent are nearer the other crescent's centre. Density-based methods like DBSCAN follow connected regions instead of distance to a point, which is why they handle arbitrary shapes and K-means cannot.

How does clustering compress data?

By replacing each data point with the centroid of its cluster, a technique called vector quantisation. Instead of storing a million distinct colour values in an image, you store 16 centroid colours and one small index per pixel saying which centroid it uses. The loss is real and bounded by how far points sit from their centre, which is exactly the quantity K-means was minimising, so the algorithm's objective and the compression quality are the same number.

Journal AI Integration February 7, 2023

AI Integration · Machine Learning

Centroid-Based Clustering: A Powerful Machine Learning Technique

Learn how centroid-based clustering algorithms like K-means partition datasets into meaningful groups based on distance metrics.

Anurag Verma

3 min read

Clustering K-Means Machine Learning Unsupervised Learning Data Science

Centroid-Based Clustering: A Powerful Machine Learning Technique

Key takeaways

A centroid is the centre of a cluster, and the algorithm moves centroids until points sit as close as possible to the one they belong to.
K-means is the default. K-medoids uses real data points as centres, which makes it far more tolerant of outliers. Fuzzy c-means lets a point belong partially to several clusters.
It suits well-separated, roughly spherical clusters where you already know or can estimate how many there are. Overlapping or irregularly shaped groups need hierarchical or density-based methods instead.
The objective is minimising within-cluster distance. Better separation between clusters is a consequence of that, not something the algorithm optimises directly.
The applications are all the same idea wearing different clothes: image segmentation, market and customer segmentation, anomaly detection, and vector quantisation for compression.
Expectation Maximisation is usually grouped separately as distribution-based rather than centroid-based, because it fits probability distributions rather than moving centre points.

Centroid-based clustering is a machine learning technique that partitions a dataset into groups of similar data points, known as clusters. This technique uses centroids, the center of each cluster, to minimize the sum of the distances between the data points and their corresponding cluster centroids. As a result, the data points are as close as possible to the center of the cluster and the inter-cluster distance is maximized.

When to Use Centroid-based Clustering for Partitioning Datasets

Centroid-based clustering is ideal for datasets with easily separable, well-defined clusters. It is also suitable when the number of clusters is known or can be easily estimated. However, it is not the best choice for datasets with overlapping clusters or non-uniform shapes. In such cases, hierarchical or density-based clustering might be more appropriate.

Different Types of Centroid-based Clustering Algorithms

Centroid-based clustering has several variations, including:

K-Means Clustering - The most commonly used centroid-based clustering algorithm that minimizes the sum of the distances between the data points and their corresponding cluster centroids.
K-Medoids Clustering - A variation of k-means that uses medoids, actual data points, as the center of each cluster instead of centroids.
Fuzzy c-Means Clustering - A variation of k-means that allows data points to belong to more than one cluster, with varying degrees of membership.
Expectation Maximization (EM) Algorithm - A model-based clustering algorithm that uses a statistical model to define the relationships between the data points and clusters.

Real-World Applications of Centroid-based Clustering for Partitioning Datasets

Centroid-based clustering has many real-world applications, including:

Image Segmentation - Dividing an image into multiple segments or regions based on color, texture, or other features using k-means or other centroid-based clustering algorithms.
Market Segmentation - Identifying smaller groups of consumers with similar needs or characteristics using k-means or other centroid-based clustering algorithms.
Customer Segmentation - Dividing a customer base into groups with similar characteristics using k-means or other centroid-based clustering algorithms.
Anomaly Detection - Identifying data points that are significantly different from the rest of the data using k-means or other centroid-based clustering algorithms.
Data Compression - Reducing the size of a dataset by replacing individual data points with their corresponding cluster centroids using k-means or other centroid-based clustering algorithms.

Conclusion

Centroid-based clustering is a powerful machine learning technique for partitioning datasets into groups of similar data points. This technique is popular and widely used, and is well-suited for datasets with well-defined clusters. It has a range of real-world applications, including image segmentation, market segmentation, customer segmentation, anomaly detection, and data compression.

GitHub link: Complete-Data-Science-Bootcamp

Main Post: Complete-Data-Science-Bootcamp

Frequently asked questions

Does K-means maximise the distance between clusters?: Not directly, and the distinction matters when you are debugging a bad result. The objective K-means actually minimises is the within-cluster sum of squared distances, meaning how tightly each group hugs its own centre. Clusters usually end up well separated as a side effect, but nothing in the algorithm penalises two centroids sitting close together. That is why K-means can happily split one real group into two when you ask for too many clusters.
When should I use K-medoids instead of K-means?: When outliers are a problem. A K-means centroid is an average, so a single extreme point drags it away from where the actual cluster sits. A K-medoid is a real data point chosen from the cluster, which cannot be pulled somewhere no data exists. The cost is speed: K-medoids is considerably more expensive to compute, so it suits smaller datasets or ones where you know the data is messy.
What does fuzzy c-means give me that K-means doesn't?: Degrees of membership rather than a hard assignment. K-means says a customer is in segment 3; fuzzy c-means says they are 60% segment 3 and 40% segment 1. That is more honest when the underlying groups genuinely overlap, which is most real customer data, and it gives you a confidence signal: points with membership split evenly across clusters are the ones sitting on a boundary and worth looking at individually.
Why is centroid-based clustering bad at non-spherical shapes?: Because assignment is by distance to a centre, which implicitly assumes clusters are roughly round blobs of similar size. Give it two interleaved crescents and it will slice straight through both, because points at the tip of one crescent are nearer the other crescent's centre. Density-based methods like DBSCAN follow connected regions instead of distance to a point, which is why they handle arbitrary shapes and K-means cannot.
How does clustering compress data?: By replacing each data point with the centroid of its cluster, a technique called vector quantisation. Instead of storing a million distinct colour values in an image, you store 16 centroid colours and one small index per pixel saying which centroid it uses. The loss is real and bounded by how far points sit from their centre, which is exactly the quantity K-means was minimising, so the algorithm's objective and the compression quality are the same number.

Newer dispatch

Sentiment Analysis of Twitter Data Using Naive Bayes Algorithm

AI Integration

Older dispatch

Clustering Algorithms: Types, Applications, and Use Cases

AI Integration

Join the conversation.

Comments are powered by GitHub Discussions. Sign in with your GitHub account to leave a comment.