[Paper Review] Two new Markov order estimators

Yuval Peres, Paul C. Shields|ArXiv.org|Jun 4, 2005

Algorithms and Data Compression18 references18 citations

TL;DR

This paper introduces two new consistent estimators for the Markov order of a finite-alphabet stochastic process using empirical entropy and conditional probability fluctuations. The entropy estimator uses recurrence times to detect when empirical entropy stabilizes, while the maximal fluctuation method identifies the order by detecting a qualitative drop in conditional probability differences; both are proven almost surely consistent as sample size increases.

ABSTRACT

We present two new methods for estimating the order (memory depth) of a finite alphabet Markov chain from observation of a sample path. One method is based on entropy estimation via recurrence times of patterns, and the other relies on a comparison of empirical conditional probabilities. The key to both methods is a qualitative change that occurs when a parameter (a candidate for the order) passes the true order. We also present extensions to order estimation for Markov random fields.

Motivation & Objective

• To develop consistent estimators for the memory depth (Markov order) of a stochastic process from a single sample path.
• To address the challenge of order estimation without prior knowledge of the true order, especially in the absence of a bound on the maximum possible order.
• To introduce methods that detect a qualitative change in behavior at the true order, leveraging asymptotic convergence properties of empirical entropy and conditional probabilities.
• To extend the proposed estimators to Markov random fields, broadening applicability beyond i.i.d. or Markov chain models.

Proposed method

• The entropy estimator method compares the empirical k-step conditional entropy $\widehat{h}_k(n)$ with a recurrence-based entropy estimator $[\ell(n)]^{-1} \log n + 2(\log n)^{-1/4}$, where $\ell(n)$ is the length of the longest repeating initial block.
• The estimator $M_n^*$ is defined as the smallest $k$ such that $\widehat{h}_k(n)$ falls below the threshold, exploiting the fact that $\widehat{h}_k(n)$ stabilizes at the true order $M$.
• The maximal fluctuation method uses a test function $\phi_m(x_{1}^{n})$ that measures the maximum deviation in empirical conditional probabilities over a sliding window of length $m$, scaled by block frequency.
• The estimator $M_n^\#$ is defined as the smallest $m$ such that $\phi_m(x_{1}^{n}) < n^{3/4}$, with $f(n) = \log \log n$ controlling the range of $m$ values tested.
• Both methods rely on a qualitative change in behavior at the true order: entropy stabilizes and fluctuations drop sharply when $k$ reaches $M$.
• Theoretical consistency is established using the ergodic theorem, large deviations, and iterated logarithm bounds for empirical entropy convergence.

Experimental results

Research questions

• RQ1 Can a consistent Markov order estimator be constructed without assuming a prior upper bound on the true order?
• RQ2 How can empirical entropy and conditional probability fluctuations be used to detect the true memory depth of a Markov process?
• RQ3 What is the behavior of empirical entropy and conditional probability differences when the candidate order $k$ is less than, equal to, or greater than the true order $M$?
• RQ4 Can the proposed estimators be extended to Markov random fields, and what theoretical guarantees hold in that setting?
• RQ5 How do the proposed estimators compare to established methods like BIC and MDL in terms of consistency and convergence behavior?

Key findings

• The entropy estimator $M_n^*$ is almost surely consistent: $\lim_{n \to \infty} M_n^*(x_1^n) = M$ for any Markov process of order $M$.
• The maximal fluctuation estimator $M_n^\#$ is also almost surely consistent, with the key insight that $\phi_m(x_{1}^{n})$ grows linearly in $n$ for $m < M$ but remains sub-polynomial for $m = M$.
• For $k < M$, $\widehat{h}_k(n)$ converges a.s. to $H_k > H$, the limiting entropy, while $\widehat{h}_M(n)$ converges to $H$, justifying the thresholding approach.
• The method using $\phi_m(x_{1}^{n})$ avoids the 'flat spot' problem common in difference-based estimators, where $H_k = H_{k+1}$ for $k < M-1$ causes premature stopping.
• The consistency of both estimators relies on the convergence rate of empirical entropy and the law of the iterated logarithm, with explicit bounds involving $\sqrt{\log \log n / n}$.
• The estimators are robust to model misspecification and do not require prior knowledge of the alphabet size or the true order, making them suitable for nonparametric settings.