[Paper Review] On the rate of convergence to equilibrium for countable ergodic Markov chains

Stefano Isola|ArXiv.org|Aug 4, 2003

Markov Chains and Monte Carlo Methods3 citations

TL;DR

This paper establishes subgeometric convergence rates of order $ n^{-d} $ for countable ergodic Markov chains, where $ d > 0 $ is the ergodic degree, using elementary generating function techniques. It links convergence behavior to spectral properties of the Markov operator on $ \ell_1 $, and provides explicit asymptotics via analytic properties of generating functions, illustrated through a renewal process model with sharp $ n^{-(\eta-1)} $ decay under $ \alpha_i \sim 1 - (1+\eta)/i $.

ABSTRACT

Using elementary methods, we prove that for a countable Markov chain $P$ of ergodic degree $d > 0$ the rate of convergence towards the stationary distribution is subgeometric of order $n^{-d}$, provided the initial distribution satisfies certain conditions of asymptotic decay. An example, modelling a renewal process and providing a markovian approximation scheme in dynamical system theory, is worked out in detail, illustrating the relationships between convergence behaviour, analytic properties of the generating functions associated to transition probabilities and spectral properties of the Markov operator $P$ on the Banach space $\ell_1$. Explicit conditions allowing to obtain the actual asymptotics for the rate of convergence are also discussed.

Motivation & Objective

• To establish sharp subgeometric convergence rates for countable ergodic Markov chains toward their stationary distribution.
• To link the rate of convergence to the ergodic degree $ d $, defined via the moments of first passage times.
• To connect convergence behavior to analytic properties of generating functions and spectral properties of the Markov operator on $ \ell_1 $.
• To provide explicit asymptotic bounds for convergence rates under suitable decay conditions on the initial distribution.
• To illustrate the theory through a detailed example modeling a renewal process and its connection to dynamical systems via Markovian approximations.

Proposed method

• Uses elementary generating function techniques to derive lower and upper bounds on the convergence rate.
• Defines the ergodic degree $ d $ as a continuous parameter controlling the number of finite moments of first passage times to a state.
• Analyzes the Markov operator $ P $ on the Banach space $ \ell_1 $, relating its spectral properties to convergence decay.
• Applies matrix-valued analytic functions in a detailed example to refine general bounds and obtain sharper asymptotics.
• Uses taboo probabilities $ f^n_{ij} $ and renewal-type quantities $ {}_k p^n_{ij} $ to characterize return and first-passage behavior.
• Establishes a correspondence between the invariant measure $ \pi $, the function $ h $ in $ L^1 $, and the density of a $ f $-invariant measure in the dynamical systems example.

Experimental results

Research questions

• RQ1 What is the precise rate of convergence to equilibrium for a countable ergodic Markov chain, and how does it depend on the ergodic degree $ d $?
• RQ2 How do the analytic properties of generating functions associated with transition probabilities relate to the spectral structure of the Markov operator on $ \ell_1 $?
• RQ3 Can the general convergence bounds be sharpened under additional regularity conditions on the transition probabilities?
• RQ4 What is the connection between the convergence rate and the first-passage time distribution in the context of renewal processes?
• RQ5 To what extent can the convergence rate be accelerated or slowed by cancellations in the test functions, as seen in the dynamical systems example?

Key findings

• For a countable ergodic Markov chain with ergodic degree $ d > 0 $, the convergence to the stationary distribution is subgeometric of order $ n^{-d} $, provided the initial distribution satisfies appropriate decay conditions.
• The convergence rate $ n^{-d} $ is sharp and can be derived using elementary generating function methods without requiring coupling or advanced probabilistic tools.
• In the renewal process example, when $ \alpha_i \sim 1 - (1+\eta)/i $ with $ \eta > 1 $, the correlation decay is $ O(n^{-(\eta-1)}) $, and this rate is asymptotically exact under non-degenerate conditions on test functions.
• The convergence rate $ n^{-(\eta-1)} $ matches the tail behavior of first entrance times into small neighborhoods of 0, indicating that the rate is fundamentally limited by recurrence statistics.
• If the test functions $ u $ and $ v $ satisfy $ \rho(u) \neq u_\infty $ and $ \rho(v) \neq v_\infty $, the correlation decay is asymptotically $ C n^{-(\eta-1)} $, confirming the sharpness of the bound.
• Cancellations in the test functions (e.g., constant functions) can accelerate convergence beyond the $ n^{-(\eta-1)} $ rate, indicating that the bound is not universal but depends on functional behavior.