[Paper Review] The Limits of Depth Reduction for Arithmetic Formulas: It's all about the top fan-in

Mrinal Kumar, Shubhangi Saraf|arXiv (Cornell University)|Nov 26, 2013

Complexity and Algorithms in Graphs3 citations

TL;DR

This paper establishes the first superpolynomial lower bounds for homogeneous depth-4 arithmetic circuits with top fan-in $ o("log n) $, using an improved depth reduction technique. It further proves that depth reduction cannot be improved when the top fan-in is $ \Omega(\log n) $, implying that the combined method of depth reduction and shifted partial derivatives is insufficient to prove lower bounds for general arithmetic formulas.

ABSTRACT

In recent years, a very exciting and promising method for proving lower bounds for arithmetic circuits has been proposed. This method combines the method of {\it depth reduction} developed in the works of Agrawal-Vinay [AV08], Koiran [Koi12] and Tavenas [Tav13], and the use of the shifted partial derivative complexity measure developed in the works of Kayal [Kay12] and Gupta et al [GKKS13a]. These results inspired a flurry of other beautiful results and strong lower bounds for various classes of arithmetic circuits, in particular a recent work of Kayal et al [KSS13] showing superpolynomial lower bounds for {\it regular} arithmetic formulas via an {\it improved depth reduction} for these formulas. It was left as an intriguing question if these methods could prove superpolynomial lower bounds for general (homogeneous) arithmetic formulas, and if so this would indeed be a breakthrough in arithmetic circuit complexity. In this paper we study the power and limitations of depth reduction and shifted partial derivatives for arithmetic formulas. We do it via studying the class of depth 4 homogeneous arithmetic circuits. We show: (1) the first {\it superpolynomial lower bounds} for the class of homogeneous depth 4 circuits with top fan-in $o(\log n)$. The core of our result is to show {\it improved depth reduction} for these circuits. (2) We show that improved depth reduction {\it is not possible} when the top fan-in is $Ω(\log n)$. In particular this shows that the depth reduction procedure of Koiran and Tavenas [Koi12, Tav13] cannot be improved even for homogeneous formulas, thus strengthening the results of Fournier et al [FLMS13] who showed that depth reduction is tight for circuits, and answering some of the main open questions of [KSS13, FLMS13].

Motivation & Objective

• To establish superpolynomial lower bounds for homogeneous depth-4 arithmetic circuits with bounded top fan-in.
• To investigate the limitations of the depth reduction technique combined with the shifted partial derivative method for general arithmetic formulas.
• To determine whether improved depth reduction is possible for homogeneous formulas when the top fan-in is $ \Omega(\log n) $.
• To resolve open questions from prior works on depth reduction tightness and lower bounds for $ \Sigma\Pi\Sigma\Pi^{[t]} $ circuits.
• To explore the implications of these results for proving lower bounds in arithmetic circuit complexity, particularly for general homogeneous formulas.

Proposed method

• Introduces an improved depth reduction procedure for homogeneous depth-4 circuits with top fan-in $ o(\log n) $, enabling stronger lower bounds.
• Applies the shifted partial derivative complexity measure to analyze the rank of partial derivatives of constructed polynomials.
• Constructs a family of polynomials $ \mathcal{P}_{t,n} $ that require $ \Sigma\Pi\Sigma\Pi^{[t/20]} $ circuits with top fan-in at least $ n^{\Omega(n/t)} $.
• Uses a linear combination of these polynomials with variable coefficients to build a single polynomial $ \mathcal{Q}_n $ that resists efficient computation by $ \Sigma\Pi\Sigma\Pi^{[a]} $ circuits for $ \omega(\log n) \leq a \leq n/800 $.
• Employs interpolation and projection arguments to show that any such circuit computing $ \mathcal{Q}_n $ must have top fan-in $ 2^{\Omega(n \log n / a)} $.
• Proves that depth reduction is tight when top fan-in is $ \Omega(\log n) $, by showing no further improvement is possible.

Experimental results

Research questions

• RQ1 Can superpolynomial lower bounds be proven for homogeneous depth-4 circuits with top fan-in $ o(\log n) $ using improved depth reduction and shifted partial derivatives?
• RQ2 Is it possible to improve the depth reduction procedure for homogeneous arithmetic formulas beyond the known bounds?
• RQ3 What are the limitations of the shifted partial derivative method when combined with depth reduction for proving lower bounds on general arithmetic formulas?
• RQ4 Can nontrivial lower bounds be established for $ \Sigma\Pi\Sigma\Pi^{[\log n]} $ circuits, which are a critical intermediate class?
• RQ5 Does the depth reduction technique of Koiran and Tavenas remain tight for homogeneous formulas, or can it be further optimized?

Key findings

• The paper establishes the first superpolynomial lower bounds for homogeneous depth-4 circuits with top fan-in $ o(\log n) $, specifically showing a lower bound of $ 2^{\Omega(n \log n / t)} $ for $ \Sigma\Pi\Sigma\Pi^{[t/20]} $ circuits.
• It proves that improved depth reduction is impossible when the top fan-in is $ \Omega(\log n) $, showing that the Koiran-Tavenas depth reduction is tight even for homogeneous formulas.
• The construction of the polynomial family $ \mathcal{Q}_n $ demonstrates that any $ \Sigma\Pi\Sigma\Pi^{[a]} $ circuit computing it must have top fan-in at least $ 2^{\Omega(n \log n / a)} $ for $ \omega(\log n) \leq a \leq n/800 $.
• The results imply that the current framework of depth reduction and shifted partial derivatives is insufficient to prove superpolynomial lower bounds for general (homogeneous) arithmetic formulas.
• The paper resolves open questions from [KSS13] and [FLMS13] by showing that depth reduction cannot be improved beyond the known bounds, even in the homogeneous case.
• A hierarchy theorem for $ \Sigma\Pi\Sigma\Pi^{[t]} $ formulas is implied, suggesting that increasing the top fan-in leads to strictly more expressive power.