[论文解读] A Direct Estimation of High Dimensional Stationary Vector Autoregressions

Fang Han, Huanran Lu+1 more|arXiv (Cornell University)|Jul 1, 2013

Statistical and numerical algorithms参考文献 43被引用 93

一句话总结

本文提出了一种新颖的高维平稳向量自回归（VAR）模型直接估计方法，通过将估计问题表述为线性规划，利用时间依赖结构实现高效的并行计算。主要贡献在于建立了理论一致性界限，将估计精度与转移矩阵的谱范数联系起来，实证结果表明该方法在参数估计和预测方面优于Lasso型估计器。

ABSTRACT

The vector autoregressive (VAR) model is a powerful tool in modeling complex time series and has been exploited in many fields. However, fitting high dimensional VAR model poses some unique challenges: On one hand, the dimensionality, caused by modeling a large number of time series and higher order autoregressive processes, is usually much higher than the time series length; On the other hand, the temporal dependence structure in the VAR model gives rise to extra theoretical challenges. In high dimensions, one popular approach is to assume the transition matrix is sparse and fit the VAR model using the "least squares" method with a lasso-type penalty. In this manuscript, we propose an alternative way in estimating the VAR model. The main idea is, via exploiting the temporal dependence structure, to formulate the estimating problem into a linear program. There is instant advantage for the proposed approach over the lasso-type estimators: The estimation equation can be decomposed into multiple sub-equations and accordingly can be efficiently solved in a parallel fashion. In addition, our method brings new theoretical insights into the VAR model analysis. So far the theoretical results developed in high dimensions (e.g., Song and Bickel (2011) and Kock and Callot (2012)) mainly pose assumptions on the design matrix of the formulated regression problems. Such conditions are indirect about the transition matrices and not transparent. In contrast, our results show that the operator norm of the transition matrices plays an important role in estimation accuracy. We provide explicit rates of convergence for both estimation and prediction. In addition, we provide thorough experiments on both synthetic and real-world equity data to show that there are empirical advantages of our method over the lasso-type estimators in both parameter estimation and forecasting.

研究动机与目标

• 解决维度d与滞后阶数p的乘积超过样本量T的高维VAR模型挑战。
• 开发一种计算高效的估计方法，利用时间依赖结构实现可并行计算。
• 通过将估计精度与转移矩阵的谱范数关联，建立关于估计一致性的新理论洞见。
• 在高维设定下，相比现有的Lasso型正则化方法，提升参数估计和预测精度。

提出的方法

• 通过利用数据中的时间依赖结构，将VAR估计问题表述为线性规划。
• 采用直接优化框架，将估计问题分解为可并行计算的子问题。
• 采用带最大范数惩罚的约束优化问题，以促进稀疏性和一致性。
• 利用高斯随机向量和矩阵范数的浓度不等式推导理论边界。
• 引入双重渐近框架，其中维度d和样本量T同时增长，且满足d/T → 0。
• 通过重新参数化和范数控制，建立线性规划与稀疏估计问题之间的等价性。

实验结果

研究问题

• RQ1 直接线性规划方法是否能在高维平稳VAR模型中实现一致估计？
• RQ2 在高维设定下，转移矩阵的谱范数如何影响估计精度？
• RQ3 在高维设定下，所提方法是否在参数估计和预测方面均优于Lasso型估计器？
• RQ4 所提估计器在何种理论条件下可实现估计和预测一致性？
• RQ5 由于其可分解结构，该方法是否可实现高效并行化？

主要发现

• 在d和T增长且满足d/T → 0的双重渐近框架下，所提方法实现了估计一致性。
• 转移矩阵A1的谱范数是估计精度的关键决定因素，一致性边界依赖于‖A1‖2。
• 以高概率成立∥bA1 − A1∥1 ≤ 4s(2λ0‖Σ−1‖1)1−q，其中s为ℓq-范数稀疏水平。
• 在合成数据和股票市场数据上的实证结果表明，该方法在参数估计和预测精度方面优于Lasso型估计器。
• 由于线性规划的可分解结构，该方法支持高效的并行计算。
• 理论边界通过高斯向量和矩阵范数的浓度不等式推导得出，对样本协方差和精度矩阵估计误差实现了高概率控制。