[Paper Review] Entropy and the fourth moment phenomenon

Ivan Nourdin, Giovanni Peccati+1 more|arXiv (Cornell University)|Apr 4, 2013

Random Matrices and Applications47 references4 citations

TL;DR

This paper introduces a novel entropy-based method using Stein factors and de Bruijn’s identity to derive quantitative total variation bounds for high-dimensional Gaussian chaos vectors. It establishes explicit, chaos-order-independent convergence rates in total variation that match smooth distance rates, resolving an open problem in the multidimensional fourth moment phenomenon for non-linear functionals of Gaussian fields.

ABSTRACT

We develop a new method for bounding the relative entropy of a random vector in terms of its Stein factors. Our approach is based on a novel representation for the score function of smoothly perturbed random variables, as well as on the de Bruijn's identity of information theory. When applied to sequences of functionals of a general Gaussian field, our results can be combined with the Carbery-Wright inequality in order to yield multidimensional entropic rates of convergence that coincide, up to a logarithmic factor, with those achievable in smooth distances (such as the 1-Wasserstein distance). In particular, our findings settle the open problem of proving a quantitative version of the multidimensional fourth moment theorem for random vectors having chaotic components, with explicit rates of convergence in total variation that are independent of the order of the associated Wiener chaoses. The results proved in the present paper are outside the scope of other existing techniques, such as for instance the multidimensional Stein's method for normal approximations.

Motivation & Objective

• To develop a new information-theoretic method for bounding relative entropy of random vectors using Stein factors and de Bruijn’s identity.
• To address the open problem of obtaining explicit, order-independent total variation convergence rates for multidimensional fourth moment theorems in chaotic functionals of Gaussian fields.
• To extend existing techniques beyond the scope of classical Stein’s method and smooth distance approximations.
• To leverage Malliavin calculus and Carbery-Wright inequalities to control entropy and total variation distances in high-dimensional, non-linear settings.
• To establish sharp, quantitative entropic central limit theorems for sequences of functionals in finite Wiener chaoses

Proposed method

• Derive a new representation of relative entropy in terms of Stein factors via a generalized integration-by-parts formula.
• Apply de Bruijn’s identity to link the derivative of relative entropy along the Ornstein-Uhlenbeck semigroup to Fisher information of perturbed vectors.
• Use Malliavin calculus to express Stein matrices for functionals in Wiener chaoses, particularly via conditional expectations of inner products of Malliavin derivatives.
• Establish uniform bounds on total variation distance using hypercontractivity and moment estimates on Stein matrices.
• Combine Carbery-Wright inequalities with entropy bounds to control the tail behavior of chaotic functionals.
• Optimize parameters in entropy and total variation estimates to derive explicit convergence rates independent of chaos order

Experimental results

Research questions

• RQ1 Can relative entropy of a high-dimensional random vector be bounded using only its Stein factors and de Bruijn’s identity?
• RQ2 What are the total variation convergence rates for sequences of functionals in finite Wiener chaoses converging to a Gaussian vector?
• RQ3 Can explicit, chaos-order-independent rates be derived for the multidimensional fourth moment theorem using entropy methods?
• RQ4 To what extent can Stein factors and de Bruijn’s formula replace classical Stein’s method in non-linear, non-i.i.d. settings?
• RQ5 How do Carbery-Wright inequalities interact with entropy-based bounds to yield sharp convergence rates in high-dimensional chaos

Key findings

• The paper establishes a new representation of relative entropy in terms of Stein factors, enabling precise control of entropy via moment and covariance information.
• A general bound on total variation distance is derived using Stein matrices and de Bruijn’s identity, valid for any random vector with finite second moments.
• For sequences of functionals in finite Wiener chaoses, the total variation distance to a Gaussian target converges at a rate of order $ (1-t)^eta $ for some $ eta > 0 $, independent of the chaos order.
• Explicit convergence rates in total variation are obtained that match those achievable in smooth distances (e.g., 1-Wasserstein), up to a logarithmic factor.
• The method yields quantitative bounds that are independent of the order $ q $ of the Wiener chaos, resolving a long-standing open problem in the multidimensional fourth moment theorem.
• The results are outside the reach of classical Stein’s method and extend to non-linear, non-i.i.d. functionals of Gaussian fields