[Paper Review] Finite Time Analysis of Vector Autoregressive Models under Linear Restrictions

Yao Zheng, Guang Cheng|arXiv (Cornell University)|Nov 26, 2018

Matrix Theory and Algorithms3 references4 citations

TL;DR

This paper develops a non-asymptotic finite-time analysis of ordinary least squares estimation in vector autoregressive models under linear restrictions, covering stable, unstable, and slightly explosive processes. It establishes that restrictions reduce estimation error not only through dimensionality reduction but also via a scale factor decreasing with the number of restrictions, with phase transitions governed by the smallest singular value of the transition matrix A.

ABSTRACT

This paper develops a unified finite-time theory for the ordinary least squares estimation of possibly unstable and even slightly explosive vector autoregressive models under linear restrictions, with the applicable region $ρ(A)\leq 1+c/n$, where $ρ(A)$ is the spectral radius of the transition matrix $A$ in the \VAR(1) representation, $n$ is the time horizon and $c>0$ is a universal constant. The linear restriction framework encompasses various existing models such as banded/network vector autoregressive models. We show that the restrictions reduce the error bounds via not only the reduced dimensionality but also a scale factor resembling the asymptotic covariance matrix of the estimator in the fixed-dimensional setup: as long as the model is correctly specified, this scale factor is decreasing in the number of restrictions. It is revealed that the phase transition from slow to fast error rate regimes is determined by the smallest singular value of $A$, a measure of the least excitable mode of the system. The minimax lower bounds are derived across different regimes. The developed non-asymptotic theory not only bridges the theoretical gap between stable and unstable regimes but precisely characterizes the effect of restrictions and its interplay with model parameters. Simulations support our theoretical results.

Motivation & Objective

• To bridge the theoretical gap between stable and unstable vector autoregressive (VAR) models by developing a unified finite-time analysis framework.
• To investigate how linear restrictions—such as banded or network-structured sparsity—affect estimation error in high-dimensional VAR models.
• To characterize the interplay between model stability (spectral radius ρ(A)) and the impact of restrictions on estimation accuracy.
• To derive minimax lower bounds across different stability regimes, including near-unit-root and slightly explosive processes.
• To establish that the smallest singular value of A governs the phase transition between slow and fast error rate regimes.

Proposed method

• Adopts a non-asymptotic, non-mixing approach based on a generalized small-ball method inspired by Mendelson (2014) and Simchowitz et al. (2018).
• Analyzes the ordinary least squares estimator under the linear restriction framework: 𝒞 vec(Aᵀ) = μ, where 𝒞 and μ are known.
• Derives error bounds using the spectral radius ρ(A) ≤ 1 + c/n as the key stability condition, allowing analysis of unstable and slightly explosive processes.
• Introduces a scale factor in the error bound that decreases with the number of restrictions, reflecting improved estimation efficiency.
• Uses empirical process theory and concentration inequalities to bound the Kullback-Leibler divergence between models, enabling minimax lower bounds.
• Applies volumetric arguments and packing arguments to construct test classes and derive lower bounds on estimation error.

Experimental results

Research questions

• RQ1 How do linear restrictions affect the finite-sample error bounds of OLS estimation in high-dimensional VAR models?
• RQ2 What is the role of the spectral radius ρ(A) in determining the estimation error rate, especially near ρ(A) = 1?
• RQ3 How does the smallest singular value of the transition matrix A govern the phase transition between slow and fast error rate regimes?
• RQ4 Can minimax lower bounds be derived that unify stable, unstable, and slightly explosive VAR processes under linear restrictions?
• RQ5 To what extent do restrictions reduce estimation error beyond just reducing dimensionality?

Key findings

• The error bound for OLS estimation under linear restrictions includes a scale factor that decreases with the number of restrictions, improving estimation accuracy beyond mere dimensionality reduction.
• The phase transition from slow to fast error rate regimes is determined by the smallest singular value of the transition matrix A, not just the spectral radius.
• For ρ(A) ∈ [1, 1 + c/n], the inverse of the growth factor γₙ(ρ̄) is bounded below by C₂/n, implying error rates of order (m/n)¹ᐟ², where m is the number of restrictions.
• When ρ(A) < 1, the error bound scales as (1 − ρ(A)²)m/n, showing improved performance as ρ(A) → 1.
• For ρ(A) > 1 + c/n, the error bound scales as (ρ(A)² − 1)m/n, with an exponential decay factor ρ(A)⁻ⁿ, indicating increasing error with explosiveness.
• Minimax lower bounds confirm that the derived error rates are statistically optimal across all stability regimes.