[论文解读] Time Series Forecasting (TSF) Using Various Deep Learning Models

Jimeng Shi, Mahek Jain+1 more|arXiv (Cornell University)|Apr 23, 2022

Air Quality Monitoring and Forecasting被引用 36

一句话总结

本文比较了 RNN、LSTM、GRU 和 Transformer 模型在北京每小时空气质量预测中的表现，考察回看窗口和预测时长对性能的影响，Transformer 往往表现最好。

ABSTRACT

Time Series Forecasting (TSF) is used to predict the target variables at a future time point based on the learning from previous time points. To keep the problem tractable, learning methods use data from a fixed length window in the past as an explicit input. In this paper, we study how the performance of predictive models change as a function of different look-back window sizes and different amounts of time to predict into the future. We also consider the performance of the recent attention-based Transformer models, which has had good success in the image processing and natural language processing domains. In all, we compare four different deep learning methods (RNN, LSTM, GRU, and Transformer) along with a baseline method. The dataset (hourly) we used is the Beijing Air Quality Dataset from the UCI website, which includes a multivariate time series of many factors measured on an hourly basis for a period of 5 years (2010-14). For each model, we also report on the relationship between the performance and the look-back window sizes and the number of predicted time points into the future. Our experiments suggest that Transformer models have the best performance with the lowest Mean Average Errors (MAE = 14.599, 23.273) and Root Mean Square Errors (RSME = 23.573, 38.131) for most of our single-step and multi-steps predictions. The best size for the look-back window to predict 1 hour into the future appears to be one day, while 2 or 4 days perform the best to predict 3 hours into the future.

研究动机与目标

• 评估随不同回看窗口大小对时间序列预测的预测精度变化。
• 评估四种深度学习模型（RNN、LSTM、GRU、Transformer）在每小时空气质量数据上的性能。
• 研究单步预测和多步预测的精度。
• 确定特定预测时长的最佳回看窗口大小。

提出的方法

• 在 Beijing Air Quality Dataset（小时数据，2010–2014）上训练并评估四种深度学习模型（RNN、LSTM、GRU、Transformer）。
• 改变回看窗口大小以研究对预测误差的影响。
• 评估单步和多步前向预测并报告 MAE 与 RMSE。
• 将基线方法与深度学习模型进行比较。
• 报告窗口大小、预测时长与模型性能之间的关系。

实验结果

研究问题

• RQ1 预测精度如何随回看窗口大小的变化而变化？
• RQ2 在 Beijing Air Quality 数据集的每小时时间序列预测中，哪种深度学习模型能获得最佳精度？
• RQ3 单步预测与多步预测在不同模型之间的比较如何？
• RQ4 1 小时和 3 小时预测的最佳回看窗口大小是什么？

主要发现

• 在所测试的架构中，Transformer 模型在单步和多步预测中均实现了最佳性能。
• Transformer 产生最低的 MAE 和 RMSE（具体数值报告为 MAE = 14.599，23.273，RMSE = 23.573，38.131）。
• 对于 1 小时前瞻预测，最佳回看窗口大约是一整天；对于 3 小时前瞻预测，2–4 天的回看窗口效果最佳。
• 相较于 Transformer，RNN、LSTM 和 GRU 在所报告的指标上通常表现不如 Transformer。
• 研究分析了在不同模型之间，回看窗口大小和预测时长如何影响性能。