[论文解读] Time Series Forecasting Using LSTM Networks: A Symbolic Approach

Steven Elsworth, Stefan Güttel|arXiv (Cornell University)|Mar 12, 2020

Time Series Analysis and Forecasting参考文献 37被引用 91

一句话总结

本论文提出将 ABBA 符号表示与 LSTM 相结合的 ABBA-LSTM，以加速训练并降低超参数敏感性，同时保持预测性能。

ABSTRACT

Machine learning methods trained on raw numerical time series data exhibit fundamental limitations such as a high sensitivity to the hyper parameters and even to the initialization of random weights. A combination of a recurrent neural network with a dimension-reducing symbolic representation is proposed and applied for the purpose of time series forecasting. It is shown that the symbolic representation can help to alleviate some of the aforementioned problems and, in addition, might allow for faster training without sacrificing the forecast performance.

研究动机与目标

• Motivate and address limitations of LSTM on raw time series data (sensitivity to hyperparameters and initialization).
• Introduce ABBA symbolic representation with a patching procedure to compress and reconstruct time series.
• Demonstrate that ABBA-LSTM can achieve comparable forecasts with faster training.
• Provide reproducible Python code and a comparative analysis with raw LSTM.

提出的方法

• Describe ABBA symbolic representation: compression into piecewise linear segments, digitization into k symbols, and patch-based reconstruction.
• Integrate ABBA with LSTM by training on symbolic sequences and using a final softmax layer for symbol prediction (ABBA-LSTM).
• Compare two training regimes (stateful vs stateless) for LSTM on time series forecasting.
• Use two-layer LSTM architectures with two initial layers of c cells, and tailor output layers for raw vs symbolic models.
• Evaluate with mean squared error for raw LSTM and categorical cross-entropy for ABBA-LSTM, with Adam optimizer and early stopping.

实验结果

研究问题

• RQ1 Can ABBA symbolic representation reduce training time while preserving forecast accuracy compared to raw LSTM?
• RQ2 Does ABBA-LSTM exhibit reduced sensitivity to hyperparameters and initialization?
• RQ3 How does stateful versus stateless training impact ABBA-LSTM and raw LSTM performance?
• RQ4 What behavioral differences emerge in forecasts when using ABBA patching versus direct numeric reconstruction?

主要发现

• ABBA-LSTM trains more easily and can achieve forecast performance similar to raw LSTM.
• Symbolic ABBA representation reduces sensitivity to hyperparameters and initial weights.
• ABBA patching yields visually more appealing forecasts by constraining outputs to observed patch shapes.
• Experiments indicate stateless training often yields larger DTW distances than stateful training, with raw LSTM generally less robust.