[Paper Review] Statistical-Computational Tradeoffs in Planted Problems and Submatrix Localization with a Growing Number of Clusters and Submatrices

Yudong Chen, Jiaming Xu|arXiv (Cornell University)|Feb 6, 2014

Sparse and Compressive Sensing Techniques81 references164 citations

TL;DR

This paper establishes a statistical-computational tradeoff framework for planted clustering and submatrix localization with a growing number of clusters/submatrices. It identifies four distinct regimes—impossible, hard, easy, and simple—based on model parameters, showing that polynomial-time algorithms achieve minimax recovery limits only in the easy and simple regimes, while the hard regime requires computationally expensive maximum likelihood estimation.

ABSTRACT

We consider two closely related problems: planted clustering and submatrix localization. The planted clustering problem assumes that a random graph is generated based on some underlying clusters of the nodes; the task is to recover these clusters given the graph. The submatrix localization problem concerns locating hidden submatrices with elevated means inside a large real-valued random matrix. Of particular interest is the setting where the number of clusters/submatrices is allowed to grow unbounded with the problem size. These formulations cover several classical models such as planted clique, planted densest subgraph, planted partition, planted coloring, and stochastic block model, which are widely used for studying community detection and clustering/bi-clustering. For both problems, we show that the space of the model parameters (cluster/submatrix size, cluster density, and submatrix mean) can be partitioned into four disjoint regions corresponding to decreasing statistical and computational complexities: (1) the \emph{impossible} regime, where all algorithms fail; (2) the \emph{hard} regime, where the computationally expensive Maximum Likelihood Estimator (MLE) succeeds; (3) the \emph{easy} regime, where the polynomial-time convexified MLE succeeds; (4) the \emph{simple} regime, where a simple counting/thresholding procedure succeeds. Moreover, we show that each of these algorithms provably fails in the previous harder regimes. Our theorems establish the minimax recovery limit, which are tight up to constants and hold with a growing number of clusters/submatrices, and provide a stronger performance guarantee than previously known for polynomial-time algorithms. Our study demonstrates the tradeoffs between statistical and computational considerations, and suggests that the minimax recovery limit may not be achievable by polynomial-time algorithms.

Motivation & Objective

• To understand the fundamental limits of recovery in planted clustering and submatrix localization when the number of clusters/submatrices grows with problem size.
• To characterize the interplay between statistical feasibility and computational efficiency in recovering hidden structures from noisy data.
• To establish a four-regime framework (impossible, hard, easy, simple) that partitions the parameter space based on recovery performance and algorithmic complexity.
• To demonstrate that polynomial-time algorithms cannot achieve the minimax recovery limit in the hard regime, highlighting a fundamental gap between statistical and computational power.
• To provide tight minimax recovery bounds that hold with high probability under general scaling of cluster size, density, and signal strength.

Proposed method

• Formalizes two core problems: planted clustering in random graphs and submatrix localization in noisy matrices with multiple disjoint submatrices.
• Introduces a four-regime classification based on model parameters: cluster size $K$, cluster density difference $p-q$, signal mean $μ$, and number of clusters $r$.
• Uses the Maximum Likelihood Estimator (MLE) as a benchmark for statistical performance in the hard regime, showing it succeeds where others fail.
• Proposes a convexified MLE that achieves minimax recovery in polynomial time in the easy regime, with provable failure in harder regimes.
• Designs a simple counting/thresholding procedure that succeeds in the simple regime, with failure guarantees in all previous regimes.
• Employs concentration inequalities (e.g., Bernstein) and combinatorial bounds on misclassified nodes to derive upper bounds on the number of equivalence classes and solution space size.

Experimental results

Research questions

• RQ1 What is the fundamental tradeoff between statistical performance and computational efficiency in recovering multiple clusters or submatrices with growing numbers?
• RQ2 In which parameter regimes can polynomial-time algorithms achieve minimax recovery, and where is the computational barrier located?
• RQ3 Can the minimax recovery limit be achieved by efficient algorithms, or is there a provable gap between statistical and computational feasibility?
• RQ4 How do the number of clusters $r$, cluster size $K$, and signal-to-noise ratio ($p-q$ or $μ$) jointly affect the recoverability of hidden structures?
• RQ5 What is the precise characterization of the boundary between regimes where simple thresholding works versus where more complex optimization is required?

Key findings

• The paper establishes a four-regime partition of the parameter space: impossible (no algorithm succeeds), hard (only MLE succeeds), easy (convexified MLE succeeds), and simple (thresholding succeeds).
• The convexified MLE achieves minimax recovery in polynomial time in the easy regime, with provable failure in the hard and impossible regimes.
• The simple thresholding procedure succeeds in the simple regime and provably fails in all previous regimes, demonstrating a sharp phase transition.
• The minimax recovery limit is tight up to constants and holds even as the number of clusters $r$ grows unbounded with $n$.
• The hard regime is computationally intractable for polynomial-time algorithms, as the MLE is the only known method that succeeds there.
• Combinatorial bounds on misclassified nodes and equivalence classes are derived using concentration and symmetry arguments, enabling tight control over solution space size.