[Paper Review] Machinery for Proving Sum-of-Squares Lower Bounds on Certification Problems

Aaron Potechin, Goutham Rajendran|arXiv (Cornell University)|Nov 9, 2020

Complexity and Algorithms in Graphs4 citations

TL;DR

This paper develops a general framework for proving Sum-of-Squares (SoS) lower bounds on certification problems by extending the pseudo-calibration technique of Barak et al. [BHK+16]. The method reduces SoS lower bound proofs to verifying three conditions—PSD mass, middle shape bounds, and intersection term bounds—enabling efficient proofs for tensor PCA, sparse PCA (Wishart model), and a variant of planted clique (planted slightly denser subgraph), achieving degree $n^\varepsilon$ SoS lower bounds that match known algorithmic thresholds.

ABSTRACT

In this paper, we construct general machinery for proving Sum-of-Squares lower bounds on certification problems by generalizing the techniques used by Barak et al. [FOCS 2016] to prove Sum-of-Squares lower bounds for planted clique. Using this machinery, we prove degree $n^{\\epsilon}$ Sum-of-Squares lower bounds for tensor PCA, the Wishart model of sparse PCA, and a variant of planted clique which we call planted slightly denser subgraph.

Motivation & Objective

• To develop a general, reusable framework for proving probabilistic Sum-of-Squares (SoS) lower bounds on certification problems.
• To generalize the pseudo-calibration technique from planted clique to broader classes of problems, such as tensor PCA and sparse PCA.
• To reduce the complexity of SoS lower bound proofs by isolating the technical analysis into verifiable conditions on coefficient matrices.
• To establish tight computational thresholds for key problems in statistics and machine learning by proving matching lower bounds to known algorithmic guarantees.
• To provide a systematic approach for proving SoS lower bounds in dense models with vertex and edge decay, extending beyond planted clique.

Proposed method

• Generalize the pseudo-calibration technique from [BHK+16] to construct candidate pseudo-expectations and moment matrices for certification problems.
• Define three core conditions—PSD mass, middle shape bounds, and intersection term bounds—on coefficient matrices that ensure the moment matrix is positive semidefinite with high probability.
• Use a shape-based decomposition of diagrams to analyze the moment matrix, leveraging symmetry and automorphism factors.
• Apply truncation error analysis by bounding contributions from high-degree diagrams using decay parameters and normalization factors.
• Leverage the structure of the coefficient matrices to decouple problem-specific analysis from the general proof machinery.
• Use a recursive bounding strategy involving left shapes, right shapes, and automorphism corrections to control the growth of terms in the moment matrix.

Experimental results

Research questions

• RQ1 Can the pseudo-calibration technique used for planted clique be generalized to other certification problems like tensor PCA and sparse PCA?
• RQ2 What minimal structural conditions on coefficient matrices ensure that the resulting moment matrix is positive semidefinite with high probability?
• RQ3 To what extent can the technical complexity of SoS lower bound proofs be reduced by abstracting away problem-specific analysis?
• RQ4 Do the SoS lower bounds proved via this machinery match the best-known algorithmic guarantees for problems like tensor PCA and sparse PCA?
• RQ5 Can this framework be extended to problems with weaker decay properties, such as Densest k-Subgraph or sparse models?

Key findings

• The paper establishes degree $n^\varepsilon$ Sum-of-Squares lower bounds for tensor PCA, matching the best-known algorithmic guarantees for this problem.
• For the Wishart model of sparse PCA, the framework yields $n^\varepsilon$ SoS lower bounds, completing the picture of approximability thresholds up to polylogarithmic factors.
• A new variant of planted clique—planted slightly denser subgraph—is shown to have $n^\varepsilon$ SoS lower bounds, extending the reach of the technique beyond standard planted clique.
• The framework reduces the proof of SoS lower bounds to verifying three structural conditions on coefficient matrices, significantly simplifying the analysis.
• The machinery successfully handles problems with both vertex and edge decay, where coefficients decay exponentially with the number of vertices and edges in the shape.
• The authors demonstrate that truncation errors in the moment matrix can be bounded using decay parameters and normalization factors, ensuring the final matrix remains positive semidefinite.