[Paper Review] Sharp minimax tests for large covariance matrices

Cristina Butucea, Rania Zgheib|arXiv (Cornell University)|Sep 4, 2014

Random Matrices and Applications15 references3 citations

TL;DR

This paper proposes a sharp minimax test for large covariance matrices using a weighted U-statistic of order 2 that adaptively balances diagonal element weighting and truncation to achieve optimal detection under high-dimensional asymptotics. It establishes exact asymptotic constants for type II and total error probabilities when n = o(1)p², proving the test attains the sharp minimax rate under ellipsoidal covariance constraints.

ABSTRACT

We consider the detection problem of correlations in a p-dimensional Gaussian vector for p large, when we observe n independent, identically distributed random vectors. We assume that the covariance matrix vary in some ellipsoid with parameter > 1=2 and total energy bounded by L > 0. We prove here both rate and sharp asymptotic results in the minimax setup. Our test procedure is a U-statistic of order 2 corrected by weighting with an optimal sequence, chosen as solution of an extremal problem. This procedure weights diagonal elements in a polynomial way and truncates the number of diagonals to take into account. We show that our test statistic has a Gaussian asymptotic behaviour under the null hypothesis and under the alternatives close to the detection boundary. Moreover, it attains the sharp asymptotic rate, i.e. with explicit asymptotic constant, for the maximal type II error and the total error probabilities, when n = o(1)p 2 . We show that sharp asymptotic lower bounds for the maximal type II error and total error probabilities under no restriction on p and n. We deduce rate asymptotic minimax results for testing the inverse of the covariance matrix.

Motivation & Objective

• To develop a minimax-optimal test for detecting correlations in high-dimensional Gaussian vectors when the sample size n and dimension p grow with n = o(1)p².
• To establish sharp asymptotic constants for the maximal type II and total error probabilities in the minimax framework under ellipsoidal covariance constraints.
• To derive lower bounds on error probabilities without restrictions on p and n, enabling rate-minimax results for testing the inverse covariance matrix.
• To design a test procedure that adapts to unknown covariance structure by solving an extremal problem for optimal weighting and truncation.

Proposed method

• The test is based on a second-order U-statistic with data-dependent weights derived from solving an extremal problem to minimize maximal error probabilities.
• The weighting scheme applies polynomial decay to diagonal elements of the sample covariance matrix and truncates the number of diagonals included.
• Asymptotic normality of the test statistic is established under both the null hypothesis and alternatives near the detection boundary.
• The procedure is designed to attain the sharp asymptotic rate, with explicit constants, for type II and total error probabilities in the high-dimensional regime.
• The method leverages the structure of ellipsoidal constraints on the covariance matrix with total energy bounded by L > 0.
• Theoretical analysis includes deriving sharp lower bounds on error probabilities to confirm the optimality of the proposed test.

Experimental results

Research questions

• RQ1 What is the sharp asymptotic rate of the maximal type II error probability for testing correlations in large-dimensional Gaussian vectors under ellipsoidal covariance constraints?
• RQ2 Can a U-statistic-based test achieve the minimax rate with explicit asymptotic constants when n = o(1)p²?
• RQ3 How do the optimal weights and truncation levels for the U-statistic depend on the underlying covariance structure and energy constraint?
• RQ4 What are the sharp lower bounds for the total error probability in the minimax setup without restrictions on p and n?
• RQ5 To what extent can the proposed test be extended to inferential problems involving the inverse covariance matrix?

Key findings

• The proposed test achieves the sharp asymptotic rate for the maximal type II error probability with an explicit asymptotic constant under the condition n = o(1)p².
• The test statistic exhibits Gaussian asymptotic distribution under both the null hypothesis and alternatives near the detection boundary.
• The optimal weighting sequence is derived as the solution to an extremal problem, balancing diagonal element contributions and truncation.
• Sharp lower bounds for the total error probability are derived, confirming the minimax optimality of the test in the high-dimensional regime.
• The results imply rate-minimax testing for the inverse covariance matrix under the same asymptotic scaling.
• The method is robust to unknown covariance structure by construction, relying only on ellipsoidal constraints and energy bounds.