[Paper Review] Anytime Hierarchical Clustering

Ömür Arslan, Daniel E. Koditschek|arXiv (Cornell University)|Apr 13, 2014

Advanced Clustering Algorithms Research39 references3 citations

TL;DR

This paper introduces an anytime hierarchical clustering algorithm that iteratively improves any initial clustering hierarchy by locally reorganizing tree structures using a homogeneity criterion compatible with common linkage functions (e.g., single, average, complete). The method guarantees finite convergence to a globally homogeneous tree—matching standard batch single linkage results—and enables scalable, decentralized, and online clustering for dynamic data via constant-time local updates.

ABSTRACT

We propose a new anytime hierarchical clustering method that iteratively transforms an arbitrary initial hierarchy on the configuration of measurements along a sequence of trees we prove for a fixed data set must terminate in a chain of nested partitions that satisfies a natural homogeneity requirement. Each recursive step re-edits the tree so as to improve a local measure of cluster homogeneity that is compatible with a number of commonly used (e.g., single, average, complete) linkage functions. As an alternative to the standard batch algorithms, we present numerical evidence to suggest that appropriate adaptations of this method can yield decentralized, scalable algorithms suitable for distributed/parallel computation of clustering hierarchies and online tracking of clustering trees applicable to large, dynamically changing databases and anomaly detection.

Motivation & Objective

• To address the limitations of batch hierarchical clustering in large, dynamically changing datasets by enabling anytime, reactive, and scalable clustering.
• To develop a local, iterative reorganization mechanism that improves cluster homogeneity without full re-clustering.
• To prove finite convergence of the algorithm to a globally homogeneous hierarchy under common linkage functions.
• To enable decentralized and parallel computation of clustering hierarchies through localized tree restructuring.
• To support online tracking of clustering trees in real-time applications such as anomaly detection.

Proposed method

• The method uses a homogeneity criterion defined via a linkage function to assess cluster quality at each node of the hierarchy.
• It applies a local tree restructuring operation—specifically, Nearest Neighbor Interchange (NNI)—to improve local homogeneity at each recursive step.
• Each NNI move is computed in constant time using sufficient statistics (e.g., mean, variance), enabling scalability.
• The algorithm is recursively applied in an anytime fashion, producing a sequence of increasingly homogeneous cluster trees.
• The process is proven to converge in finite time for a class of linkages including single, complete, average, minimax, and Ward’s linkage.
• The framework supports incremental updates and dynamic dissimilarity measures, enabling online adaptation.

Experimental results

Research questions

• RQ1 Can a local, iterative reorganization of hierarchical clustering trees achieve global homogeneity and finite convergence?
• RQ2 How does the anytime clustering method compare to standard batch clustering in terms of accuracy and convergence?
• RQ3 Can the method be adapted for decentralized, parallel, or online computation in large-scale or dynamic data environments?
• RQ4 What is the relationship between the homogeneity criterion and standard linkage-based clustering outcomes?
• RQ5 Can the method be extended to support simultaneous distance metric learning during clustering?

Key findings

• The anytime clustering method converges in finite time to a globally homogeneous hierarchy for a broad class of linkage functions, including single, complete, average, minimax, and Ward’s linkage.
• For single linkage, the method converges to the unique tree produced by standard batch agglomerative clustering, regardless of initial hierarchy.
• Experimental results show that anytime and incremental clustering perform comparably to batch agglomerative clustering for average and Ward’s linkages, with cophenetic correlation coefficients close to the ground truth.
• Performance is lower for complete and minimax linkages due to overestimation of inter-cluster dissimilarities, but still effective in practice.
• The method exhibits quadratic growth in iterations for single linkage and potentially linear growth for other linkages, suggesting scalability with further optimization.
• Incremental clustering outperforms anytime clustering due to better initialization, but both methods significantly improve over random initial trees.