Trilingual clustering tutorial with Mermaid diagrams, comparison tables, and Python 3.9+ runnable demos for K-Means++, Fuzzy C-Means, Hierarchical, DBSCAN, OPTI
Trilingual clustering tutorial with Mermaid diagrams, comparison tables, and Python 3.9+ runnable demos for K-Means++, Fuzzy C-Means, Hierarchical, DBSCAN, OPTI
…Language: English | 简体中文 | 日本語
This repository is a refreshed clustering-algorithm learning project. It keeps the first four classic algorithms as readable hand-written Python demos, adds two widely used modern extensions, and turns the old blog-style notes into a GitHub-friendly, runnable reference.
--seed, --samples, --output, and --show.Clustering is the task of grouping unlabeled samples by similarity. The key difference between clustering algorithms is not just speed, but the very definition of what a "cluster" means: a tight group around a center, a connected dense region, a merge path in a hierarchy, or a partition in a similarity graph.
flowchart TD
A["Raw unlabeled samples"] --> B{"What makes points belong together?"}
B --> C["Nearest prototype or center
K-Means++, Fuzzy C-Means"]
B --> D["Repeatedly merge closest groups
Agglomerative Hierarchical"]
B --> E["Dense reachable neighborhoods
DBSCAN, OPTICS"]
B --> F["Graph neighborhood structure
Spectral Clustering"]python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txtRun any demo directly:
python3 k_means_plus_plus.py
python3 fuzzy_c_means.py
python3 agglomerative_hierarchical.py
python3 dbscan.py
python3 optics.py
python3 spectral_clustering.pyUseful options shared by all scripts:
--seed: controls reproducibility.--samples: controls the dataset size.--output: saves the figure to a custom path.--show: displays the figure after saving it.Run the smoke tests:
pytest| File | Purpose |
|---|---|
| clustering_utils.py | Shared dataset generation, plotting helpers, validation helpers, and k-means++ initialization. |
| k_means_plus_plus.py | Hand-written K-Means++ demo for convex blob-shaped clusters. |
| fuzzy_c_means.py | Hand-written fuzzy c-means demo for soft memberships. |
| agglomerative_hierarchical.py | Hand-written agglomerative hierarchical clustering demo. |
| dbscan.py | Hand-written DBSCAN demo for noise-aware density clustering. |
| optics.py | OPTICS demo with a reachability plot, using scikit-learn. |
| spectral_clustering.py | Spectral clustering demo for non-convex shapes, using scikit-learn. |
| requirements.txt | Minimal dependencies for scripts and tests. |
| Algorithm | Family | Need k in advance? |
Handles noise? | Works well on non-convex shapes? | Typical complexity | Best fit |
|---|---|---|---|---|---|---|
| K-Means++ | Prototype / partitioning | Yes | No | Usually no | Time O(nkt), space O(n + k) |
Fast baseline when clusters are compact and roughly spherical |
| Fuzzy C-Means | Soft partitioning | Yes | No | Usually no | Time O(nkt), space O(nk) |
When soft memberships are more informative than hard assignments |
| Agglomerative Hierarchical | Hierarchical | No fixed k during merging, but you choose a cut level |
Limited | Sometimes | Demo time O(n^3), space O(n^2) |
Small datasets where merge history matters |
| DBSCAN | Density-based | No | Yes | Yes | Demo time O(n^2), space O(n^2) |
Irregular clusters with outliers and unknown cluster count |
| OPTICS | Density-based ordering | No | Yes | Yes | Often near O(n log n) with indexing, worst O(n^2) |
Datasets with different local densities |
| Spectral Clustering | Graph / partitioning | Yes | No explicit noise model | Yes | Time often O(n^3), space O(n^2) |
Non-convex manifolds and graph-like similarity structure |
| If your data looks like this | Best first algorithm | Why it is a good starting point |
|---|---|---|
| Compact blob-like groups and a known cluster count | K-Means++ | Fast, stable, and easy to explain |
| Samples overlap and hard labels hide useful uncertainty | Fuzzy C-Means | Keeps soft memberships instead of a single forced label |
| You want to inspect coarse-to-fine grouping structure | Agglomerative Hierarchical | The merge path itself is part of the result |
| Clusters are irregular and outliers matter | DBSCAN | Density expansion naturally separates noise |
| Density changes a lot across regions | OPTICS | More flexible than DBSCAN under one global threshold |
| Rings, moons, or graph-shaped neighborhoods dominate | Spectral Clustering | Uses graph structure instead of center geometry |
| Algorithm | Most important knobs | If you increase it | If you decrease it |
|---|---|---|---|
| K-Means++ | --clusters, --tolerance |
More clusters can split groups; larger tolerance stops earlier | Fewer clusters merge groups; smaller tolerance runs longer |
| Fuzzy C-Means | --clusters, --fuzziness |
Larger fuzziness makes memberships softer | Smaller fuzziness makes assignments harder |
| Agglomerative | --clusters, --linkage |
More target clusters keeps finer structure | Fewer target clusters force more merging |
| DBSCAN | --eps, --min-samples |
Larger eps or smaller min-samples tends to merge more points |
Smaller eps or larger min-samples tends to split clusters and create more noise |
| OPTICS | --xi, --min-cluster-size |
Larger minimum size favors broader, more conservative clusters | Smaller minimum size allows finer clusters but can fragment the result |
| Spectral | --clusters, --neighbors |
More neighbors smooth the graph more strongly | Fewer neighbors keep local structure sharper but can disconnect the graph |
Core idea
K-Means++ is still the classic k-means objective, but it replaces random seeding with a smarter initialization strategy. By spreading the initial centers apart, it usually reaches better local optima and converges faster than naive random starts.
Algorithm steps
Process map
flowchart TD
A["Initialize centers with k-means++"] --> B["Assign every point to the nearest center"]
B --> C["Recompute each center as the cluster mean"]
C --> D{"Centers moved less than tolerance?"}
D -- "No" --> B
D -- "Yes" --> E["Return labels, centers, and center trace"]Key parameters
--clusters: number of centers to fit.--max-iter: optimization steps before forcing a stop.--tolerance: convergence threshold for center movement.Strengths
Limitations
Use when
Avoid when
Complexity
O(nkt) where n is the sample count, k is the cluster count, and t is the number of iterations.O(n + k) for the main state in this demo.Run
python3 k_means_plus_plus.py --samples 320 --clusters 4 --output k_means_plus_plus.pngCore source snippet
…Key parameters
--clusters: number of fuzzy centers.--fuzziness: controls how soft the memberships are. Values closer to 1 behave more like hard clustering.--max-iter: optimization budget.--tolerance: convergence threshold.Strengths
Limitations
Use when
Avoid when
Complexity
O(nkt) with a larger constant than K-Means.O(nk) because the membership matrix is stored explicitly.Run
python3 fuzzy_c_means.py --samples 320 --clusters 4 --fuzziness 2.0 --output fuzzy_c_means.pngCore source snippet
…Source: fuzzy_c_means.py
Core idea
Agglomerative clustering starts with one cluster per point and repeatedly merges the closest pair of clusters. Instead of committing to one level immediately, it builds structure from fine detail to coarse detail.
Algorithm steps
Process map
flowchart TD
A["Start with one cluster per point"] --> B["Measure distances between clusters"]
B --> C["Merge the closest two clusters"]
C --> D{"Target number of clusters reached?"}
D -- "No" --> B
D -- "Yes" --> E["Cut the hierarchy and output labels"]Key parameters
--clusters: number of clusters to keep at the cut level.--linkage: single, average, or complete.Strengths
No open issues yet, or sync has not completed.