[Paper Review] Bootstrap Central Limit Theorem for Chains of Infinite Order via Markov Approximations

Pierre Collet, Denise Duarte+1 more|ArXiv.org|May 11, 2005

Bayesian Methods and Mixture Models13 references3 citations

TL;DR

This paper introduces a bootstrap resampling method for chains of infinite order by leveraging canonical Markov approximations. It establishes a Bootstrap Central Limit Theorem for the sample mean by showing that, under exponential memory decay, the bootstrap distribution of the empirical mean converges to a normal distribution as the order $k$ and number of blocks $m_k$ diverge with a controlled relationship.

ABSTRACT

We present a new approach to the bootstrap for chains of infinite order taking values on a finite alphabet. It is based on a sequential Bootstrap Central Limit Theorem for the sequence of canonical Markov approximations of the chain of infinite order. Combined with previous results on the rate of approximation this leads to a Central Limit Theorem for the bootstrapped estimator of the sample mean which is the main result of this paper.

Motivation & Objective

• To develop a valid bootstrap procedure for stochastic processes with long-range dependence, specifically chains of infinite order.
• To address the challenge of resampling dependent data where standard i.i.d. bootstrap fails due to temporal dependence.
• To establish a central limit theorem for the bootstrap estimator of the sample mean in such processes.
• To bridge the gap between the behavior of infinite-order chains and their finite-order Markov approximations for statistical inference.
• To ensure the bootstrap distribution asymptotically matches the true sampling distribution of the sample mean.

Proposed method

• Constructs a sequence of canonical Markov chains of order $k$ to approximate the infinite-order chain.
• Uses excursions between successive occurrences of a $k$-symbol initial string as i.i.d. blocks for resampling.
• Applies maximal coupling to couple the original chain and its Markov approximation for joint construction.
• Employs the $φ$-mixing property and rate bounds from prior work to control approximation error.
• Uses block resampling via i.i.d. uniform selection of blocks to generate bootstrap samples.
• Establishes convergence of the bootstrap variance and distribution by controlling the probability of path mismatch between original and approximated chains.

Experimental results

Research questions

• RQ1 Can a bootstrap procedure be constructed for chains of infinite order that preserves asymptotic validity?
• RQ2 How can Markov approximations of increasing order $k$ be used to enable block-based resampling in dependent processes?
• RQ3 Under what conditions does the bootstrap distribution of the sample mean converge to a normal distribution for infinite-order chains?
• RQ4 What rate of convergence is achievable for the bootstrap estimator when approximating infinite-order processes?
• RQ5 How does the coupling between the original chain and its Markov approximation affect the validity of the bootstrap?

Key findings

• The bootstrap distribution of the sample mean converges in law to a normal distribution under the stated conditions.
• The convergence holds when the order $k$ and number of blocks $m_k$ diverge with $c > 18 \log \delta^{-1}$, ensuring exponential decay of memory.
• The key technical result is that the difference between the second moments of the bootstrap estimators of the original and approximated chains vanishes as $k \to \infty$.
• The probability of path mismatch between the original chain and its Markov approximation up to time $R_{m_k}(k)$ tends to zero as $k \to \infty$.
• The ratio of bootstrap variances $\sigma^{[k]*}/\sigma^*_{k}$ converges in probability to 1, ensuring consistency of the variance estimator.
• The bootstrap central limit theorem is established by coupling the bootstrap of the original chain with that of its Markov approximation and showing asymptotic equivalence.