Intiution

Intiution#

Hierarchical clustering is about grouping similar points into clusters in a nested, tree-like structure.

  • Instead of pre-defining the number of clusters (like K-Means), HC creates a hierarchy of clusters.

  • The hierarchy is often visualized as a dendrogram, which shows how clusters merge (or split) step by step.

2. Intuition: Agglomerative (Bottom-Up) HC#

  1. Start with each point as its own cluster

    • Imagine every data point is a leaf on a tree.

  2. Merge the closest clusters iteratively

    • “Closest” is determined by a distance metric (Euclidean, Manhattan, etc.) and linkage method (single, complete, average, Ward).

    • Merge these points to form a branch of the tree.

  3. Repeat until all points are merged into one cluster

    • The final cluster is the root of the tree.

  4. Dendrogram shows the process

    • Height of a merge represents the distance between clusters.

    • Cutting the dendrogram at a certain height gives a specific number of clusters.

Analogy:

  • Imagine clustering friends based on how close they are:

    • First, best friends stick together (small clusters).

    • Then, groups of friends merge into larger social circles.

    • Finally, everyone forms one big network.

3. Intuition: Divisive (Top-Down) HC#

  1. Start with all points in one cluster.

  2. Recursively split clusters based on distance or variance.

  3. Continue splitting until each point is its own cluster.

This is less common in practice because it is computationally expensive.

4. Key Points in HC Intuition#

Concept

Intuition

Distance metric

Measures “closeness” between points or clusters

Linkage method

Decides how clusters are merged (min distance, max distance, average, variance)

Dendrogram

Tree showing the merging/splitting process

Cutting the dendrogram

Choosing the number of clusters visually based on desired similarity

Nested structure

HC naturally captures sub-clusters within larger clusters

5. Visual Example (Conceptual)#

  • Imagine 10 points on a line:

Points:  A   B   C       D   E   F       G   H   I   J

  • Agglomerative HC merges closest points:

    1. Merge A & B, D & E, G & H … → small clusters

    2. Merge clusters based on distance → bigger clusters

    3. Merge all → root cluster

  • Dendrogram height shows distance at which clusters merge.

Takeaway

Hierarchical clustering is intuitive because it’s like building a tree of relationships:

  • Closest points merge first → small branches

  • Similar branches merge → larger branches

  • Final root contains all points

It gives a visual, interpretable view of cluster structure, especially useful for exploring nested relationships.

Contents