[Paper Review] Limit theorems for invariant distributions

Morgane Austern, Peter Orbanz|arXiv (Cornell University)|Jun 27, 2018

Bayesian Methods and Mixture Models35 references3 citations

TL;DR

This paper establishes central limit theorems, Berry-Esseen bounds, and concentration inequalities for invariant distribution estimators based on averaging over transformation groups, using ergodic theory and Stein's method. It proves asymptotic normality of empirical entropy for a broad class of processes under mixing and group growth conditions.

ABSTRACT

A distributional symmetry is invariance of a distribution under a group of transformations. Exchangeability and stationarity are examples. We explain that a result of ergodic theory provides a law of large numbers: If the group satisfies suitable conditions, expectations can be estimated by averaging over subsets of transformations, and these estimators are strongly consistent. We show that, if a mixing condition holds, the averages also satisfy a central limit theorem, a Berry-Esseen bound, and concentration. These are extended further to apply to triangular arrays, to randomly subsampled averages, and to a generalization of U-statistics. As applications, we obtain new results on exchangeability, random fields, network models, and a class of marked point processes. We also establish asymptotic normality of the empirical entropy for a large class of processes. Some known results are recovered as special cases, and can hence be interpreted as an outcome of symmetry. The proofs adapt Stein's method.

Motivation & Objective

• To develop a general framework for consistent estimation of expectations under group-invariant distributions.
• To establish central limit theorems and Berry-Esseen bounds for estimators based on averaging over transformation groups.
• To extend results to triangular arrays, randomly subsampled averages, and generalized U-statistics.
• To prove asymptotic normality of empirical entropy for invariant processes under mixing and group growth conditions.
• To unify and generalize known results in exchangeability, random fields, network models, and point processes through symmetry.

Proposed method

• Uses Lindenstrauss's pointwise ergodic theorem as a foundation for strong consistency of group-averaged estimators.
• Applies Stein's method to derive Berry-Esseen bounds and concentration inequalities for group-averaged functionals.
• Introduces a mixing condition defined via coefficients α(t,k) for k-tuples of group elements to control dependence.
• Employs a left-invariant total order on groups to define predictive dependence and control approximation error in entropy estimation.
• Uses tempered Følner sequences to ensure asymptotic equivalence of group averages over expanding subsets.
• Derives asymptotic variance η² as a sum of covariances between predictive log-likelihoods at different group elements.

Experimental results

Research questions

• RQ1 Under what conditions is the group-averaged estimator Fn(f,X) asymptotically normal for invariant distributions?
• RQ2 How can Berry-Esseen bounds be derived for invariant estimators using Stein's method?
• RQ3 What mixing conditions ensure asymptotic normality of empirical entropy in invariant processes?
• RQ4 How does the choice of group structure (e.g., polynomial growth, torsion-free) affect the convergence properties of the estimators?
• RQ5 Can known results in exchangeability and network models be recovered as special cases of this general framework?

Key findings

• The group-averaged estimator Fn(f,X) is strongly consistent under G-invariance and tempered Følner sequences.
• A central limit theorem holds for Fn(f,X) under a mixing condition involving α(t,k) and a growth condition on the group.
• A Berry-Esseen bound is established with rate depending on the mixing coefficient and group growth.
• Asymptotic normality of the empirical entropy hn(λ,X) is proven under a summability condition on ρm and α(i−m,|Bm|).
• The asymptotic variance η² is expressed as a sum of covariances between predictive log-likelihoods, independent of the choice of order under left-invariance.
• The framework recovers known results in exchangeability, random fields, and network models as special cases.