[Paper Review] Towards an Axiomatic Approach to Hierarchical Clustering of Measures

Philipp Thomann, Ingo Steinwart+1 more|arXiv (Cornell University)|Aug 15, 2015

Bayesian Methods and Mixture Models25 references6 citations

TL;DR

This paper proposes an axiomatic framework for hierarchical clustering of probability measures without relying on metrics or similarity measures. It defines clustering via user-specified clusterings on elementary measures, proving that under additivity and continuity axioms, a unique hierarchical clustering is induced on a broad class of distributions, including those without densities. The key contribution is a general, mathematically rigorous foundation for clustering that extends to infinite-sample regimes and mixed Hausdorff-dimensional measures.

ABSTRACT

We propose some axioms for hierarchical clustering of probability measures and investigate their ramifications. The basic idea is to let the user stipulate the clusters for some elementary measures. This is done without the need of any notion of metric, similarity or dissimilarity. Our main results then show that for each suitable choice of user-defined clustering on elementary measures we obtain a unique notion of clustering on a large set of distributions satisfying a set of additivity and continuity axioms. We illustrate the developed theory by numerous examples including some with and some without a density.

Motivation & Objective

• To develop a general, axiomatic framework for hierarchical clustering of probability measures that does not depend on metrics or similarity measures.
• To formalize clustering as a function on measures by specifying user-defined clusterings on a base set of elementary measures.
• To establish conditions under which such a clustering function is uniquely determined by axioms of additivity and continuity.
• To extend the theory to complex distributions, including those without densities and those with mixed Hausdorff dimensions.
• To provide a rigorous foundation for infinite-sample clustering that supports consistency and learning rate analysis.

Proposed method

• Define a set of axioms—Axioms 1, 2, and 3—governing hierarchical clustering functions on probability measures, emphasizing additivity and continuity.
• Use the user’s specification of clusters on a base class of elementary measures as the starting point, without requiring metric or similarity notions.
• Introduce the concept of limit structures via Definition 18 to ensure uniqueness of the clustering extension.
• Prove Theorem 20, which establishes sufficient conditions for the existence of a unique limit structure, enabling the construction of a unique additive and continuous clustering.
• Apply the framework to examples involving density level sets and mixed Hausdorff-dimensional measures, showing consistency with known clustering notions.
• Leverage measure-theoretic tools such as inner regularity, support definitions, and Radon-Nikodym derivatives to formalize continuity and additivity.

Experimental results

Research questions

• RQ1 Can a hierarchical clustering function be uniquely defined on a large class of probability measures based solely on user-specified clusters for elementary measures?
• RQ2 What axioms—specifically additivity and continuity—are necessary and sufficient to ensure uniqueness of the clustering extension?
• RQ3 How does this axiomatic framework relate to existing clustering notions such as density level set clustering or mode-based clustering?
• RQ4 Can the framework handle distributions without densities, such as singular measures or mixed Hausdorff-dimensional measures?
• RQ5 What conditions ensure that the limit structure required for uniqueness exists in general measure spaces?

Key findings

• For any suitable choice of user-defined clustering on elementary measures, there exists a unique hierarchical clustering on a large class of probability measures satisfying the axioms of additivity and continuity.
• Theorem 20 provides a criterion (via Definition 18) for the existence of a unique limit structure, which is essential for constructing the unique clustering function.
• Theorem 21 establishes that under the conditions of Theorem 20, a unique additive and continuous clustering function exists on the entire space of measures.
• The framework includes and generalizes density level set clustering, showing that such clustering arises naturally as a special case of the axiomatic approach.
• The theory applies to measures on Radon spaces, including Polish spaces and separable Banach spaces with weak topology, and supports measures with mixed Hausdorff dimensions.
• The support and absolute continuity properties of measures (e.g., via Radon-Nikodym derivatives) are formally used to ensure continuity and consistency of the clustering function.