[Paper Review] Bernstein's inequalities for general Markov chains

Bai Jiang, Qiang Sun+1 more|arXiv (Cornell University)|May 28, 2018

Control Systems and Identification27 references19 citations

TL;DR

This paper establishes sharp Bernstein-type concentration inequalities for general Markov chains—covering general-state-space and non-reversible chains—by bounding the moment generating function of partial sums using a perturbed Markov operator's operator norm. The key contribution is a novel convex decomposition of the moment generating function bound into a classical Bernstein term and a dependence-adjusted term, yielding tighter deviation bounds than prior work, especially in low-variance, high-dependence settings.

ABSTRACT

We establish Bernstein's inequalities for functions of general (general-state-space and possibly non-reversible) Markov chains. These inequalities achieve sharp variance proxies and encompass the classical Bernstein inequality for independent random variables as special cases. The key analysis lies in bounding the operator norm of a perturbed Markov transition kernel by the exponential of sum of two convex functions. One coincides with what delivers the classical Bernstein inequality, and the other reflects the influence of the Markov dependence. A convex analysis on these two functions then derives our Bernstein inequalities. As applications, we apply our Bernstein inequalities to the Markov chain Monte Carlo integral estimation problem and the robust mean estimation problem with Markov-dependent samples, and achieve tight deviation bounds that previous inequalities can not.

Motivation & Objective

• To extend classical Bernstein inequalities to general Markov chains, including non-reversible and general-state-space chains.
• To derive concentration inequalities with sharp variance proxies that reduce to classical results under independence.
• To overcome the challenge of infinite-dimensional Markov operators in general state spaces using discretization and Léon-Perron operators.
• To apply the new inequalities to MCMC integral estimation and robust mean estimation under Markov dependence.
• To achieve tighter non-asymptotic confidence intervals and improved deviation bounds compared to prior Hoeffding-type inequalities.

Proposed method

• Use the Cramér-Chernoff method to derive tail bounds from moment generating function (mgf) bounds on sums of functions of Markov chains.
• Introduce a Léon-Perron operator as a convex majorant of the Markov transition kernel to handle infinite-dimensional operators.
• Apply Kato’s perturbation theory to expand the operator norm of the perturbed kernel $ E^{tf/2} ilde{P} E^{tf/2} $ as a power series in $ t $, where $ ilde{P} $ is the Léon-Perron operator.
• Discretize the function $ f $ as $ ilde{f}_k $, reducing the infinite-dimensional problem to finite-dimensional approximations that preserve sharpness.
• Decompose the mgf bound into two convex functions: $ g_1(t) = rac{ ext{Var}(f)}{c^2}(e^{tc} - 1 - tc) $ (classical Bernstein) and $ g_2(t) = rac{ ext{Var}(f) ho t^2}{1 - ho - 5ct} $ (dependence correction), both convex in $ t $.
• Use the Moreau-Rockafellar formula to lower-bound the Fenchel conjugate of $ g_1(t) + g_2(t) $, ensuring no loss in variance proxy sharpness.

Experimental results

Research questions

• RQ1 Can Bernstein-type concentration inequalities be extended to general-state-space, non-reversible Markov chains with sharp variance proxies?
• RQ2 How can the moment generating function of sums of functions of Markov chains be bounded in infinite-dimensional settings?
• RQ3 What is the role of spectral properties and operator perturbation theory in deriving tighter concentration bounds under Markov dependence?
• RQ4 How do the new inequalities compare to existing Hoeffding-type bounds in MCMC and robust mean estimation?
• RQ5 Can the dependence structure of Markov chains be explicitly quantified in concentration inequalities via a convex decomposition of the mgf?

Key findings

• The proposed Bernstein inequalities achieve sharp variance proxies and reduce exactly to the classical Bernstein inequality for independent random variables.
• The moment generating function of $ rac{1}{n} extstyleigsum_{i=1}^n f(X_i) $ is bounded by $ ext{exp}ig(n ig( g_1(t) + g_2(t) ig)ig) $, where $ g_1(t) $ corresponds to the classical Bernstein term and $ g_2(t) $ captures Markov dependence.
• In MCMC integral estimation, the non-asymptotic confidence intervals derived from the new inequalities are at least 30% tighter than those from Paulin (2015).
• For robust mean estimation under Markov dependence, the new inequalities yield diminishing upper bounds, unlike Hoeffding-type bounds (e.g., Fan et al., 2021), which only give non-diminishing bounds.
• The method preserves sharpness in the variance proxy by using the Moreau-Rockafellar formula to handle the conjugate of the sum of convex functions.
• The use of Léon-Perron operators and function discretization allows rigorous extension of finite-state-space techniques to general-state-space chains without loss of sharpness.