[Paper Review] Minimax Optimal Variable Clustering in G-models via Cord

Florentina Bunea, Christophe Giraud+1 more|arXiv (Cornell University)|Aug 8, 2015

Bayesian Methods and Mixture Models1 references8 citations

TL;DR

This paper introduces CORD, a minimax optimal method for variable clustering in G-models using copula-based similarity, achieving exact partition recovery at a rate of $\sqrt{\log p / n}$, irrespective of cluster count or size, with polynomial-time computation and guaranteed identifiability via $G$-exchangeable and $G$-block covariance models.

ABSTRACT

The goal of variable clustering is to partition a random vector ${\bf X} \in R^p$ in sub-groups of similar probabilistic behavior. Popular methods such as hierarchical clustering or $K$-means are algorithmic procedures applied to observations on ${\bf X}$, while no population level target is defined prior to estimation. We take a different view in this paper, where we propose and investigate model based variable clustering. We consider three models, of increasing level of complexity, termed generically $G$-models, with $G$ standing for the partition to be estimated. Motivated by the potential lack of identifiability of the $G$-latent models, which are currently used in problems involving variable clustering, we introduce two new classes of models, the $G$-exchangeable and the $G$-block covariance models. We show that both classes are identifiable, for any distribution of ${\bf X}$. Our focus is on clusters that are invariant with respect to unknown monotone transformations of the data, and that can be estimated in a computationally feasible manner. Both desiderata can be met if the clusters correspond to blocks in the copula correlation matrix of ${\bf X}$, assumed to have a Gaussian copula distribution. This motivates the introduction of a new similarity metric for cluster membership, CORD, and a homonymous method for cluster estimation. Central to our work is the derivation of the minimax value of the CORD cluster separation for exact partition recovery. We obtained the surprising result that this value is of order $\sqrt{{\log (p)}/{n}}$, irrespective of the number of clusters, or of the size of the smallest cluster. Our new procedure, CORD, available on CRAN, achieves this bound, is easy to implement and has computational complexity that is polynomial in $p$.

Motivation & Objective

• To address the lack of population-level targets in existing variable clustering methods by proposing model-based clustering with identifiable models.
• To develop a clustering method invariant to monotone transformations of the data, ensuring robustness across data scales.
• To achieve exact partition recovery with minimax optimal performance, independent of the number or size of clusters.
• To introduce a computationally feasible method with polynomial complexity in $p$, suitable for high-dimensional settings.
• To ensure identifiability of the clustering model by introducing $G$-exchangeable and $G$-block covariance models.

Proposed method

• Proposes $G$-exchangeable and $G$-block covariance models as identifiable alternatives to traditional latent variable models in variable clustering.
• Defines a new similarity metric, CORD, based on the copula correlation matrix under a Gaussian copula assumption.
• Uses the copula correlation matrix to identify clusters as blocks, ensuring invariance under monotone transformations.
• Derives the minimax optimal cluster separation threshold for exact partition recovery as $\sqrt{\log p / n}$.
• Develops a polynomial-time algorithm for CORD that achieves the minimax rate.
• Employs a population-level modeling approach, avoiding reliance on algorithmic procedures like $K$-means or hierarchical clustering applied to data samples.

Experimental results

Research questions

• RQ1 What is the minimax optimal rate for exact variable clustering recovery in $G$-models under a Gaussian copula assumption?
• RQ2 Can a clustering method be both invariant to monotone transformations and identifiable across all distributions of $\mathbf{X}$?
• RQ3 Is it possible to achieve exact partition recovery with a computational complexity that is polynomial in $p$, regardless of cluster structure?
• RQ4 How does the minimax rate depend on $p$ and $n$, and does it depend on the number or size of clusters?
• RQ5 Can identifiability be guaranteed in variable clustering models without imposing strong parametric assumptions?

Key findings

• The minimax optimal cluster separation for exact partition recovery is $\sqrt{\log p / n}$, independent of the number of clusters or the size of the smallest cluster.
• The CORD method achieves this minimax rate, demonstrating optimal statistical performance in variable clustering.
• The $G$-exchangeable and $G$-block covariance models are identifiable for any distribution of $\mathbf{X}$, resolving prior identifiability issues in $G$-latent models.
• CORD is computationally efficient, with polynomial-time complexity in $p$, enabling scalability to high-dimensional settings.
• The method is invariant to unknown monotone transformations of the data, ensuring robustness in real-world applications.
• CORD is available on CRAN, enabling practical deployment and reproducibility in statistical software.