[Paper Review] Bisect and Conquer: Hierarchical Clustering via Max-Uncut Bisection

Sara Ahmadian, Vaggos Chatziafratis+5 more|arXiv (Cornell University)|Dec 15, 2019

Advanced Clustering Algorithms Research37 references4 citations

TL;DR

This paper presents a novel hierarchical clustering algorithm that combines Max-Uncut Bisection and Average-Linkage to achieve a 0.4246 approximation ratio for the maximization dual of Dasgupta's objective, significantly improving upon the prior best of 0.3363. The method uses Max-Uncut Bisection as a black-box subroutine, enabling both theoretical guarantees and practical deployment with graceful degradation in performance based on the quality of the bisection subroutine.

ABSTRACT

Hierarchical Clustering is an unsupervised data analysis method which has been widely used for decades. Despite its popularity, it had an underdeveloped analytical foundation and to address this, Dasgupta recently introduced an optimization viewpoint of hierarchical clustering with pairwise similarity information that spurred a line of work shedding light on old algorithms (e.g., Average-Linkage), but also designing new algorithms. Here, for the maximization dual of Dasgupta's objective (introduced by Moseley-Wang), we present polynomial-time .4246 approximation algorithms that use Max-Uncut Bisection as a subroutine. The previous best worst-case approximation factor in polynomial time was .336, improving only slightly over Average-Linkage which achieves 1/3. Finally, we complement our positive results by providing APX-hardness (even for 0-1 similarities), under the Small Set Expansion hypothesis.

Motivation & Objective

• To address the lack of a strong analytical foundation for hierarchical clustering by building on Dasgupta's objective and its maximization dual.
• To improve the worst-case approximation ratio for hierarchical clustering beyond the previous best of 0.3363 achieved via semidefinite programming.
• To develop a practical and analyzable algorithm that leverages existing Max-Uncut Bisection solvers as a black-box subroutine.
• To establish APX-hardness for the problem under the Small Set Expansion hypothesis, providing theoretical limits on approximation.
• To demonstrate that combining Max-Uncut Bisection with Average-Linkage yields superior performance compared to prior convex relaxation-based methods.

Proposed method

• The algorithm uses Max-Uncut Bisection to partition the data into two balanced subsets at the root of the dendrogram.
• Each subset is recursively processed using Average-Linkage to build the subtrees, ensuring hierarchical clustering at all levels.
• The objective value is analyzed by decomposing edge contributions into intra-subtree and inter-subtree components based on the least common ancestor.
• A key technical component is the derivation of a lower bound on the expected weight of uncut edges in the bisection, parameterized by the fraction of edges cut.
• The approximation ratio is derived by relating the performance of the bisection subroutine (with factor ρ ≈ 0.8776) to the overall hierarchical clustering objective.
• The analysis shows that the algorithm’s approximation factor degrades gracefully with the quality of the Max-Uncut Bisection subroutine.

Experimental results

Research questions

• RQ1 Can a hierarchical clustering algorithm achieve a better worst-case approximation ratio than the prior best of 0.3363 for the maximization dual of Dasgupta’s objective?
• RQ2 Can Max-Uncut Bisection be effectively used as a subroutine in a hierarchical clustering framework to yield strong theoretical guarantees?
• RQ3 Does combining Max-Uncut Bisection with Average-Linkage lead to a more practical and analyzable algorithm than prior semidefinite programming-based approaches?
• RQ4 What is the best possible approximation ratio achievable under the Small Set Expansion hypothesis, and is the problem APX-hard?
• RQ5 How does the approximation factor of the algorithm degrade as a function of the approximation quality of the Max-Uncut Bisection subroutine?

Key findings

• The proposed algorithm achieves a 0.4246 approximation ratio for the maximization dual of Dasgupta’s hierarchical clustering objective, significantly improving upon the prior best of 0.3363.
• The algorithm uses Max-Uncut Bisection as a black-box subroutine, and its approximation ratio degrades gracefully with the quality of this subroutine.
• The analysis shows that even with an optimal bisection (ρ = 1), the best achievable approximation ratio would only reach 4/9 ≈ 0.444, indicating that the current bound is nearly tight.
• The paper establishes APX-hardness for the problem under the Small Set Expansion hypothesis, even for 0-1 similarity matrices, indicating strong theoretical limits on approximation.
• The method is both theoretically sound and practically deployable, as high-quality Max-Uncut Bisection solvers are readily available in the literature.
• The algorithm outperforms prior methods that relied on complex semidefinite programming relaxations, offering a simpler and more interpretable framework.