[论文解读] AR-Net: A simple Auto-Regressive Neural Network for time-series

Oskar Triebe, Nikolay Laptev+1 more|arXiv (Cornell University)|Nov 27, 2019

Time Series Analysis and Forecasting参考文献 22被引用 50

一句话总结

AR-Net 提出了一种基于神经网络的方法，使用前馈网络来模仿 Classic-AR 系数，实现与 AR 阶数线性尺度的扩展，并能够实现稀疏、可解释的长程自回归模型。

ABSTRACT

In this paper we present a new framework for time-series modeling that combines the best of traditional statistical models and neural networks. We focus on time-series with long-range dependencies, needed for monitoring fine granularity data (e.g. minutes, seconds, milliseconds), prevalent in operational use-cases. Traditional models, such as auto-regression fitted with least squares (Classic-AR) can model time-series with a concise and interpretable model. When dealing with long-range dependencies, Classic-AR models can become intractably slow to fit for large data. Recently, sequence-to-sequence models, such as Recurrent Neural Networks, which were originally intended for natural language processing, have become popular for time-series. However, they can be overly complex for typical time-series data and lack interpretability. A scalable and interpretable model is needed to bridge the statistical and deep learning-based approaches. As a first step towards this goal, we propose modelling AR-process dynamics using a feed-forward neural network approach, termed AR-Net. We show that AR-Net is as interpretable as Classic-AR but also scales to long-range dependencies. Our results lead to three major conclusions: First, AR-Net learns identical AR-coefficients as Classic-AR, thus being equally interpretable. Second, the computational complexity with respect to the order of the AR process, is linear for AR-Net as compared to a quadratic for Classic-AR. This makes it possible to model long-range dependencies within fine granularity data. Third, by introducing regularization, AR-Net automatically selects and learns sparse AR-coefficients. This eliminates the need to know the exact order of the AR-process and allows to learn sparse weights for a model with long-range dependencies.

研究动机与目标

• 桥接传统的 AR 模型与神经网络，以建模具有长程依赖的时间序列。
• 为高阶自回归过程提供一个可解释、可扩展的 Classic-AR 的替代方案。
• 使 AR 系数学习实现稀疏化，以避免需要精确的 AR 阶。
• 证明 AR-Net 可以在学习 AR 系数方面与 Classic-AR 完全一致，同时提供计算上的优势。

提出的方法

• 用一个前馈神经网络来建模 AR 动力学，其第一层的权重对应 AR 系数。
• 使用均方误差作为训练目标，以与 Classic-AR 拟合保持一致。
• 引入面向稀疏性的正则化，促使对无关系数进行零化。
• 在来自带噪 AR 过程的合成数据上将 AR-Net 与 Classic-AR 进行比较。
• 评估两者的系数精度（sTPE）和一步预测均方误差。
• 讨论 AR-Net 与 Classic-AR 的线性与二次复杂度在 AR 阶拟合中的差异。

实验结果

研究问题

• RQ1 当在同一数据上拟合时，AR-Net 能否在 AR 系数方面与 Classic-AR 完全一致？
• RQ2 AR-Net 是否随 AR 阶线性扩展并在可控计算成本下实现长程依赖？
• RQ3 正则化能否在不牺牲预测精度的情况下诱导出稀疏的 AR 系数？
• RQ4 在小数据集上或当真实 AR 阶未知时，AR-Net 的表现如何？
• RQ5 随着 p 增大，AR-Net 相对于最小二乘 AR 的计算优势是什么？

主要发现

• AR-Net 在 AR 系数上与 Classic-AR 完全一致，并获得相似的一步预测均方误差。
• AR-Net 的训练复杂度随 AR 阶 p 线性扩展，而 Classic-AR 线性？二次扩展，使 AR-Net 在长程依赖场景更可取。
• 正则化使 AR 系数稀疏化，即使真实稀疏性非常高也有效（实验中稀疏度高达 0.003）。
• 在稀疏设置中，AR-Net 的 sTPE 精度优于 Classic-AR，而预测性能保持近似一致。
• AR-Net 在小数据集和强噪声条件下仍然有效，具有可比的 MSE，但系数稀疏性优于 Classic-AR。