[Paper Review] High-Dimensional Matched Subspace Detection When Data are Missing

Laura Balzano, Bejamin Recht+1 more|arXiv (Cornell University)|Feb 3, 2010

Sparse and Compressive Sensing Techniques7 references11 citations

TL;DR

This paper proposes a robust matched subspace detection method for high-dimensional signals with missing data, using a projection-based energy estimator on incomplete observations. It proves that reliable detection is possible with only $ O(r\log r) $ random samples when the subspace is incoherent, enabling scalable anomaly detection in large-scale systems like sensor networks or hyperspectral imaging.

ABSTRACT

We consider the problem of deciding whether a highly incomplete signal lies within a given subspace. This problem, Matched Subspace Detection, is a classical, well-studied problem when the signal is completely observed. High- dimensional testing problems in which it may be prohibitive or impossible to obtain a complete observation motivate this work. The signal is represented as a vector in R^n, but we only observe m << n of its elements. We show that reliable detection is possible, under mild incoherence conditions, as long as m is slightly greater than the dimension of the subspace in question.

Motivation & Objective

• Address the challenge of detecting whether a high-dimensional signal lies within a low-dimensional subspace when only a small, incomplete subset of its entries are observed.
• Overcome the fundamental flaw of zero-filled estimators that fail to preserve subspace structure even when the true signal lies in the subspace.
• Develop a statistically sound estimator of signal energy in the subspace using only partial observations, ensuring consistency under mild incoherence assumptions.
• Establish theoretical bounds on estimation error and detection reliability, showing that detection performance degrades gracefully with missing data.
• Demonstrate that detection is possible with far fewer samples than the ambient dimension, making the method scalable to large-scale applications.

Proposed method

• Propose a projection-based energy estimator $ \|P_{S_\Omega}v_\Omega\|_2^2 $, where $ S_\Omega $ is the subspace spanned by the observed rows of the subspace basis matrix $ U $, to estimate the true signal energy in $ S $.
• Use the pseudoinverse of the observed submatrix $ U_\Omega $ to compute projections, ensuring that if $ v \in S $, the residual $ \|v_\Omega - P_{S_\Omega}v_\Omega\|_2^2 $ is zero.
• Apply the Noncommutative Bernstein Inequality to bound the deviation of the empirical Gram matrix $ \sum_{k \in \Omega} U_k U_k^T $ from its expectation $ \frac{m}{n}I_r $, ensuring invertibility with high probability.
• Introduce a coherence condition $ \mu(S) \leq \frac{n}{r} $ to control the maximum projection of standard basis vectors onto the subspace, ensuring stable estimation.
• Derive high-probability bounds on the estimation error of the energy in the subspace, showing that the observed energy scales linearly with the true energy outside the subspace.
• Use uniform sampling with replacement of signal entries to ensure statistical fairness and enable concentration inequalities for theoretical guarantees.

Experimental results

Research questions

• RQ1 Can reliable matched subspace detection be achieved when only a small number of entries of a high-dimensional signal are observed?
• RQ2 How many random samples are sufficient to reliably estimate the energy of a signal in a low-dimensional subspace when data are missing?
• RQ3 What conditions on the subspace structure (e.g., coherence) ensure that the observed energy in the subspace is a consistent estimator of the true energy?
• RQ4 How does the performance of the proposed estimator compare to naive zero-filled approaches in terms of bias and variance?
• RQ5 Can theoretical bounds be derived to quantify the probability of correct detection under missing data, even when the number of samples is much smaller than the ambient dimension?

Key findings

• The proposed estimator $ \|P_{S_\Omega}v_\Omega\|_2^2 $ is consistent and significantly outperforms the zero-filled estimator $ \|P_S\widetilde{v}\|_2^2 $, which can produce large errors even when $ v \in S $.
• Under uniform sampling with replacement, the estimator achieves high-probability bounds on estimation error, with the probability of large deviation decaying exponentially in the number of samples.
• The method guarantees reliable detection with $ m = O(r\log r) $ samples, where $ r $ is the subspace dimension, under mild incoherence conditions.
• Theoretical analysis shows that the residual energy $ \|v_\Omega - P_{S_\Omega}v_\Omega\|_2^2 $ concentrates around $ \frac{m}{n}\|v - P_S v\|_2^2 $, ensuring accurate detection of energy outside the subspace.
• The probability of error in detecting signal energy outside $ S $ is bounded by $ \exp\left(-\frac{\beta^2 \|y\|_2^2}{2n\|y\|_\infty^2}\right) $, which decays rapidly with signal-to-noise ratio and sample size.
• The invertibility of the empirical Gram matrix $ \sum_{k \in \Omega} U_k U_k^T $ is guaranteed with high probability when $ m \gg r \log r $, ensuring stable estimation.