[Paper Review] Aggregated test of independence based on HSIC measures

Anouar Meynaoui, Béatrice Laurent+2 more|arXiv (Cornell University)|Feb 15, 2019

Random Matrices and Applications36 references4 citations

TL;DR

This paper proposes an aggregated HSIC-based test for independence between random vectors that adaptively selects bandwidths across multiple Gaussian kernels, avoiding the need for manual kernel choice. The method achieves minimax adaptive optimality over Sobolev balls, with sharp non-asymptotic separation rates proven via theoretical bounds and validated through numerical studies.

ABSTRACT

Dependence measures based on reproducing kernel Hilbert spaces, also known as Hilbert-Schmidt Independence Criterion and denoted HSIC, are widely used to statistically decide whether or not two random vectors are dependent. Recently, non-parametric HSIC-based statistical tests of independence have been performed. However, these tests lead to the question of the choice of the kernels associated to the HSIC. In particular, there is as yet no method to objectively select specific kernels with theoretical guarantees in terms of first and second kind errors. One of the main contributions of this work is to develop a new HSIC-based aggregated procedure which avoids such a kernel choice, and to provide theoretical guarantees for this procedure. To achieve this, we first introduce non-asymptotic single tests based on Gaussian kernels with a given bandwidth, which are of prescribed level $\\alpha \\in (0,1)$. From a theoretical point of view, we upper-bound their uniform separation rate of testing over Sobolev and Nikol'skii balls. Then, we aggregate several single tests, and obtain similar upper-bounds for the uniform separation rate of the aggregated procedure over the same regularity spaces. Another main contribution is that we provide a lower-bound for the non-asymptotic minimax separation rate of testing over Sobolev balls, and deduce that the aggregated procedure is adaptive in the minimax sense over such regularity spaces. Finally, from a practical point of view, we perform numerical studies in order to assess the efficiency of our aggregated procedure and compare it to existing independence tests in the literature.

Motivation & Objective

• To develop a non-asymptotic, adaptive test of independence that avoids the challenging choice of kernel bandwidth in HSIC-based methods.
• To establish theoretical guarantees for the aggregated procedure, including sharp upper bounds on the uniform separation rate over Sobolev balls.
• To derive a lower bound on the non-asymptotic minimax separation rate, proving that the aggregated test is minimax adaptive over Sobolev regularity spaces.
• To evaluate the practical performance of the aggregated test through numerical studies, comparing it to existing HSIC-based and non-HSIC independence tests.

Proposed method

• Construct single HSIC tests using Gaussian kernels with fixed bandwidths, each controlling the Type I error at a prescribed level α.
• Aggregate multiple single tests across a range of bandwidths using a data-driven combination rule to improve robustness and adaptivity.
• Employ permutation methods to approximate the null distribution of the aggregated test statistic, ensuring valid size control.
• Derive non-asymptotic upper bounds on the uniform separation rate of the aggregated procedure over Sobolev balls of smoothness δ.
• Establish a lower bound on the non-asymptotic minimax separation rate over the same Sobolev classes, proving adaptivity in the minimax sense.
• Use concentration inequalities and moment bounds on U-statistics to control the expectation of the squared test statistic under the null.

Experimental results

Research questions

• RQ1 Can an aggregated HSIC test be constructed that adapts to unknown smoothness in the dependence structure without requiring manual bandwidth selection?
• RQ2 What is the non-asymptotic uniform separation rate of the aggregated HSIC test over Sobolev balls of smoothness δ?
• RQ3 Is the proposed aggregated test minimax optimal in the non-asymptotic sense over Sobolev balls?
• RQ4 How does the performance of the aggregated test compare to existing HSIC-based and non-HSIC independence tests in finite samples?
• RQ5 Can theoretical guarantees on the separation rate be established using moment methods and U-statistic theory?

Key findings

• The aggregated HSIC test achieves a uniform separation rate that matches the non-asymptotic minimax rate up to a constant factor over Sobolev balls, proving minimax adaptivity.
• The upper bound on the uniform separation rate is of order n^{-(2δ+p+q)/(4δ+2p+2q)} for smoothness δ and dimension (p,q), matching the known minimax rate.
• A lower bound on the minimax separation rate is derived, confirming that the aggregated procedure is optimal in the minimax sense over Sobolev balls.
• Theoretical analysis shows that the expectation of the squared test statistic under the null is uniformly bounded, ensuring stability and size control.
• Numerical studies demonstrate that the aggregated test outperforms standard HSIC tests with fixed bandwidths and other state-of-the-art independence tests in various scenarios.
• The method is robust across different smoothness levels and dimensions, with no need for tuning beyond bandwidth selection, which is automated via aggregation.