[Paper Review] Minimax Estimation of Linear Functions of Eigenvectors in the Face of Small Eigen-Gaps

Gen Li, Changxiao Cai+2 more|arXiv (Cornell University)|Apr 7, 2021

Random Matrices and Applications60 references4 citations

TL;DR

This paper develops de-biased estimators for linear functions of eigenvectors in matrix denoising and principal component analysis under Gaussian noise, achieving minimax optimality even with small eigen-gaps. The proposed method corrects bias in plug-in estimators via data-driven, sample-splitting-free procedures, enabling accurate estimation of fine-grained eigenvector features in high-dimensional settings.

ABSTRACT

Eigenvector perturbation analysis plays a vital role in various data science applications. A large body of prior works, however, focused on establishing $\ell_{2}$ eigenvector perturbation bounds, which are often highly inadequate in addressing tasks that rely on fine-grained behavior of an eigenvector. This paper makes progress on this by studying the perturbation of linear functions of an unknown eigenvector. Focusing on two fundamental problems -- matrix denoising and principal component analysis -- in the presence of Gaussian noise, we develop a suite of statistical theory that characterizes the perturbation of arbitrary linear functions of an unknown eigenvector. In order to mitigate a non-negligible bias issue inherent to the natural ``plug-in'' estimator, we develop de-biased estimators that (1) achieve minimax lower bounds for a family of scenarios (modulo some logarithmic factor), and (2) can be computed in a data-driven manner without sample splitting. Noteworthily, the proposed estimators are nearly minimax optimal even when the associated eigen-gap is {\em substantially smaller} than what is required in prior statistical theory.

Motivation & Objective

• To address the inadequacy of existing ℓ₂ eigenvector perturbation theory for estimating fine-grained features of eigenvectors.
• To develop bias-corrected estimators for linear functions of unknown eigenvectors in matrix denoising and PCA.
• To achieve minimax optimality in estimation error even when eigen-gaps are small, challenging prior assumptions.
• To construct estimators that are computable in a data-driven manner without sample splitting.
• To provide a unified statistical theory for linear functionals of eigenvectors under low-rank matrix models with i.i.d. Gaussian noise.

Proposed method

• Proposes a de-biased estimator that corrects the systematic bias inherent in the natural plug-in estimator of linear functions of eigenvectors.
• Derives theoretical bounds using truncated matrix Bernstein inequalities to control tail behavior of random matrix products.
• Employs a master theorem framework to characterize the bias and variance of the de-biased estimator under general noise models.
• Introduces a data-driven bias correction mechanism that avoids sample splitting while maintaining minimax optimality.
• Applies eigenvalue and eigenvector perturbation theory to derive concentration bounds for the empirical eigenvectors.
• Establishes minimax lower bounds to prove optimality of the proposed estimators, modulo logarithmic factors.

Experimental results

Research questions

• RQ1 Can we achieve minimax optimal estimation of linear functions of eigenvectors when the eigen-gap is small, violating classical assumptions?
• RQ2 How can we correct the bias in plug-in estimators for linear functionals of eigenvectors without relying on sample splitting?
• RQ3 What is the fundamental statistical limit (minimax risk) for estimating linear functions of eigenvectors in matrix denoising and PCA?
• RQ4 How do the proposed de-biased estimators behave under high-dimensional asymptotics with small eigen-gaps?
• RQ5 Can the proposed method be applied to both matrix denoising and PCA with a unified theoretical framework?

Key findings

• The proposed de-biased estimator achieves minimax optimality for linear functionals of eigenvectors, up to a logarithmic factor, even when the eigen-gap is small.
• The minimax lower bound for the estimation error of linear functions of eigenvectors is derived, establishing the fundamental statistical limit.
• The plug-in estimator is shown to suffer from a non-negligible bias that prevents minimax optimality, especially in low eigen-gap regimes.
• The de-biased estimator is computable in a data-driven way without sample splitting, enabling practical deployment.
• Theoretical analysis confirms that the estimator’s risk scales as O(√(pr log n) + √(p log³n) + log²n), matching the minimax lower bound up to logarithmic factors.
• The method is robust to weak spectral separation, extending the applicability of eigenvector estimation beyond classical eigen-gap assumptions.