[Paper Review] Limit results for distributed estimation of invariant subspaces in multiple networks inference and PCA

Runbing Zheng, Minh Tang|arXiv (Cornell University)|Jun 9, 2022

Random Matrices and Applications4 citations

TL;DR

This paper proposes a distributed estimation method for the leading singular vectors of matrices sharing invariant subspaces, using averaging of projection matrices followed by eigendecomposition. It establishes uniform $ℓ_{2\to\infty}$ error bounds and shows that row-wise fluctuations of the estimated singular vectors are asymptotically normal, enabling valid inference in multiple networks and PCA settings under sub-Gaussian or random graph models.

ABSTRACT

Several statistical problems, such as multiple heterogeneous graph analysis, distributed PCA, integrative data analysis, and simultaneous dimension reduction of images, can involve a collection of $m$ matrices whose leading subspaces $U^{(i)}$ consist of a shared subspace $U_c$ and individual subspaces $U_s^{(i)}$. We consider a distributed estimation procedure that first obtains $\hat U^{(i)}$ as the leading singular vectors for each observed noisy matrix, then computes the leading left singular vectors of the concatenated matrix $[\hat U^{(1)}|\hat U^{(2)}|\dots|\hat U^{(m)}]$ as $\hat U_c$, and finally computes the leading singular vectors of the projection of each $\hat U^{(i)}$ onto the orthogonal complement of $\hat U_c$ as $\hat U_s^{(i)}$. In this paper, we provide a framework for deriving limit results for such distributed estimation procedures, including expansions of estimation errors in both common and individual subspaces and their asymptotically normal approximations. We apply this framework specifically to (1) parameter estimation for multiple heterogeneous random graphs with shared subspaces, and (2) distributed PCA for independent sub-Gaussian random vectors with spiked covariance structures. Leveraging these results, we also consider a two-sample test for the null hypothesis that a pair of random graphs have the same edge probabilities, and present a test statistic whose limiting distribution converges to a central (resp., non-central) $χ^2$ distribution under the null (resp., local alternative) hypothesis.

Motivation & Objective

• To develop a theoretically grounded distributed estimation procedure for shared leading singular subspaces across multiple matrices.
• To address limitations in existing methods that lack uniform error bounds and distributional approximations for distributed estimators.
• To establish asymptotic normality of row-wise fluctuations in the estimated singular vectors under sub-Gaussian and random graph models.
• To enable valid statistical inference, such as two-sample hypothesis testing, in distributed settings with heterogeneous network or data structures.
• To provide a general framework applicable to distributed PCA, multiple network inference (e.g., COSIE model), and integrative data analysis.

Proposed method

• The method estimates projection matrices onto leading singular subspaces for each individual matrix at distributed nodes.
• It computes the average of these projection matrices across all nodes.
• The leading eigenvectors of the averaged projection matrix are used as the final estimate of the invariant subspace.
• The analysis relies on concentration inequalities and spectral norm bounds for random matrix products under sub-Gaussian and random graph assumptions.
• It derives uniform $ℓ_{2\to\infty}$ error bounds by controlling the $2\to\infty$ norm of residual terms in the perturbation expansion.
• It establishes asymptotic normality of row-wise estimation errors via a central limit theorem for the fluctuation terms.

Experimental results

Research questions

• RQ1 Can we establish uniform $ℓ_{2\to\infty}$ error bounds for distributed estimation of invariant subspaces in multiple matrices?
• RQ2 Do the row-wise fluctuations of the estimated singular vectors converge to a normal distribution under sub-Gaussian or random graph models?
• RQ3 How does the proposed distributed estimator compare to centralized estimators in terms of estimation accuracy and inference validity?
• RQ4 Can the method support valid two-sample hypothesis testing for edge probability equality in random graphs?
• RQ5 What is the impact of network heterogeneity and sample size on the convergence rate of the distributed estimator?

Key findings

• The row-wise fluctuations of the estimated singular vectors are asymptotically normally distributed around the true singular vectors under the proposed distributed algorithm.
• The uniform $ℓ_{2\to\infty}$ error bound for the estimator is of order $d^{1/2}n^{-1/2}D^{-\gamma}\log^{1/2}n$, which vanishes as $n,D \to \infty$.
• The two-sample test statistic for edge probability equality converges in distribution to a non-central $\chi^2$ under local alternatives and to a central $\chi^2$ under the null hypothesis.
• The leading-order term in the perturbation expansion of the estimator matches the form derived from the central limit theorem, validating the normal approximation.
• The lower-order residual terms in the expansion are negligible in both spectral and $2\to\infty$ norms under the assumed model conditions.
• The method achieves consistent estimation and valid inference even when the underlying matrices (e.g., adjacency matrices or data matrices) are heterogeneous across nodes.