[Paper Review] Computational Lower Bounds for Sparse PCA

Quentin Berthet, Philippe Rigollet|arXiv (Cornell University)|Apr 3, 2013

Sparse and Compressive Sensing Techniques42 references64 citations

TL;DR

This paper establishes computational lower bounds for sparse principal component analysis (PCA) under the assumption that the planted clique problem is hard in average-case settings. It shows that no computationally efficient method can detect weaker signals than those detected by a semidefinite programming-based test, implying a fundamental statistical price to pay for computational efficiency in sparse PCA detection.

ABSTRACT

In the context of sparse principal component detection, we bring evidence towards the existence of a statistical price to pay for computational efficiency. We measure the performance of a test by the smallest signal strength that it can detect and we propose a computationally efficient method based on semidefinite programming. We also prove that the statistical performance of this test cannot be strictly improved by any computationally efficient method. Our results can be viewed as complexity theoretic lower bounds conditionally on the assumptions that some instances of the planted clique problem cannot be solved in randomized polynomial time.

Motivation & Objective

• To investigate whether computationally efficient methods for sparse PCA detection incur a statistical performance penalty compared to optimal, but intractable, methods.
• To formalize a notion of optimality that accounts for computational constraints in high-dimensional sparse detection problems.
• To establish that the detection threshold achieved by a semidefinite programming relaxation cannot be improved by any polynomial-time method, under plausible complexity-theoretic assumptions.
• To extend existing results on computational limits in high-dimensional statistics by linking sparse PCA detection to the average-case hardness of the planted clique problem.
• To provide a conditional lower bound on the minimal signal strength detectable in polynomial time, using reductions from a well-known hard problem in average-case complexity.

Proposed method

• Proposes a semidefinite programming relaxation for sparse PCA detection, building on the method of d'Aspremont et al. (2007), and analyzes its detection threshold.
• Introduces a novel reduction from the planted clique problem to the sparse PCA detection problem, showing that improved detection performance would imply a randomized polynomial-time algorithm for planted clique.
• Uses a randomized polynomial-time transformation (blow-up map) to embed a planted clique instance into a sparse PCA testing problem.
• Applies concentration inequalities and total variation bounds to control the statistical behavior of the transformed problem under null and alternative hypotheses.
• Employs a coupling argument to show that the distribution of the transformed data under the alternative is statistically close to a product measure, enabling the use of hypothesis testing lower bounds.
• Derives a conditional lower bound on the detection threshold by assuming the average-case hardness of the planted clique problem, using a conjecture widely accepted in complexity theory and cryptography.

Experimental results

Research questions

• RQ1 Can the detection performance of any computationally efficient method for sparse PCA exceed that of the semidefinite programming relaxation?
• RQ2 Is there a fundamental gap between the optimal detection threshold and what is achievable in polynomial time for sparse PCA detection?
• RQ3 To what extent does the average-case hardness of the planted clique problem imply computational limits in high-dimensional statistical inference?
• RQ4 Can a reduction from the planted clique problem to sparse PCA detection establish a tight lower bound on the minimal detectable signal strength under polynomial-time constraints?
• RQ5 Does the existence of a statistical price for computational efficiency in sparse PCA detection hold under standard complexity-theoretic assumptions?

Key findings

• The detection threshold achieved by the semidefinite programming relaxation for sparse PCA is unimprovable by any computationally efficient method, assuming the average-case hardness of the planted clique problem.
• The optimal detection rate for polynomial-time tests is bounded below by $ \sqrt{k^\alpha / n} $ and above by $ \sqrt{k^2 \log d / n} $, with $ \alpha \in [1,2) $, under the condition $ k \leq n^{1/(4-\alpha)} $.
• The gap between the optimal detection threshold $ \theta^* $ and the threshold achievable in polynomial time $ \theta^\circ $ is of order $ \sqrt{k} $, indicating a significant statistical cost for computational efficiency.
• The reduction from the planted clique problem to sparse PCA detection establishes that improving detection performance beyond the SDP threshold would yield a randomized polynomial-time algorithm for planted clique, which is widely believed to be impossible.
• The results are conditional on a standard conjecture in average-case complexity: that the planted clique problem cannot be solved in randomized polynomial time for certain parameter regimes.
• The framework applies to general distributions and extends prior results on matrix and sparse signal detection, providing a broader theoretical foundation for computational limits in high-dimensional statistics.