[Paper Review] The average number of critical rank-one approximations to a tensor

Jan Draisma, Emil Horobeţ|arXiv (Cornell University)|Aug 15, 2014

Tensor decomposition and applications16 references4 citations

TL;DR

This paper computes the expected number of real critical rank-one approximations to a random tensor drawn from a Gaussian distribution, using a novel formula that reduces the high-dimensional integral to a lower-dimensional one via random matrix theory. The key contribution is a closed-form expression involving integrals over symmetric matrices, with explicit numerical results showing the average number of critical points grows rapidly with tensor size, especially in symmetric cases.

ABSTRACT

Motivated by the many potential applications of low-rank multi-way tensor approximations, we set out to count the rank-one tensors that are critical points of the distance function to a general tensor v. As this count depends on v, we average over v drawn from a Gaussian distribution, and find formulas that relates this average to problems in random matrix theory.

Motivation & Objective

• To determine the expected number of real critical rank-one approximations to a general tensor under a Gaussian distribution.
• To address the challenge that the number of critical points varies with the tensor, making a single count infeasible.
• To derive a formula that reduces the high-dimensional integral over the tensor space to a lower-dimensional integral using random matrix theory.
• To compare real critical point counts with the known complex count from [FO12], revealing differences in behavior and growth.

Proposed method

• Derive an expression for the expected number of real critical points by integrating the count of critical points over the tensor space with respect to a Gaussian measure.
• Reduce the original N-dimensional integral (N = ∏ni) to a lower-dimensional integral over a space of dimension 1 + ∑i<j(ni−1)(nj−1), parameterized by w0 and matrices Cij.
• Use tools from random matrix theory, particularly the distribution of absolute determinants of Gaussian matrices, to evaluate the reduced integral.
• Apply the Kuznetsov trace formula and results from [Mui82] to evaluate the integral involving eigenvalues and Vandermonde determinants.
• Leverage symmetry and invariance properties to simplify the integral over the space of symmetric matrices.
• Numerically estimate the integrals using Monte Carlo sampling with doubling sample sizes until convergence to 10−4 relative error.

Experimental results

Research questions

• RQ1 What is the expected number of real critical rank-one approximations to a random tensor drawn from a standard Gaussian distribution?
• RQ2 How does the average number of real critical points compare to the known complex count from [FO12]?
• RQ3 Can the high-dimensional integral over the tensor space be reduced to a tractable lower-dimensional integral using random matrix theory?
• RQ4 Does the average number of critical points stabilize or exhibit a specific asymptotic behavior as tensor dimensions grow?
• RQ5 What geometric or algebraic structure underlies the observed discrepancy between real and complex critical point counts?

Key findings

• The expected number of real critical rank-one approximations to a Gaussian tensor is given by a formula involving an integral over a space of dimension 1 + ∑i<j(ni−1)(nj−1), with a prefactor depending on the n_i and gamma functions.
• For symmetric tensors, the integral is over a much smaller domain, enabling accurate numerical evaluation with Mathematica, yielding precise estimates such as 9.3951 for p=4, n=4.
• In the 2×2×2 case, the average number of real critical points is approximately 4.287, significantly less than the complex count of 6.
• For 2×2×k formats, the average count stabilizes around 5.5–5.6 for k≥4, suggesting a saturation effect not seen in the complex case.
• The average count grows rapidly with tensor size: for 2×2×10, it reaches ~1.843×10^4, far exceeding the complex count of 3,628,800.
• The results suggest that real critical point counts do not stabilize but continue to grow, contrasting with the complex case where the count stabilizes beyond the boundary format.