[论文解读] TSMixer: An All-MLP Architecture for Time Series Forecasting

Sian Chen, Chunliang Li+3 more|arXiv (Cornell University)|Mar 10, 2023

Time Series Analysis and Forecasting被引用 119

一句话总结

TSMixer 提出了一种全MLP架构，通过交替时间混合和特征混合模块来预测多变量时间序列，在标准基准上取得具有竞争力的结果，在大规模 M5 数据上通过利用跨变量和辅助信息实现最先进的性能。

ABSTRACT

Real-world time-series datasets are often multivariate with complex dynamics. To capture this complexity, high capacity architectures like recurrent- or attention-based sequential deep learning models have become popular. However, recent work demonstrates that simple univariate linear models can outperform such deep learning models on several commonly used academic benchmarks. Extending them, in this paper, we investigate the capabilities of linear models for time-series forecasting and present Time-Series Mixer (TSMixer), a novel architecture designed by stacking multi-layer perceptrons (MLPs). TSMixer is based on mixing operations along both the time and feature dimensions to extract information efficiently. On popular academic benchmarks, the simple-to-implement TSMixer is comparable to specialized state-of-the-art models that leverage the inductive biases of specific benchmarks. On the challenging and large scale M5 benchmark, a real-world retail dataset, TSMixer demonstrates superior performance compared to the state-of-the-art alternatives. Our results underline the importance of efficiently utilizing cross-variate and auxiliary information for improving the performance of time series forecasting. We present various analyses to shed light into the capabilities of TSMixer. The design paradigms utilized in TSMixer are expected to open new horizons for deep learning-based time series forecasting. The implementation is available at https://github.com/google-research/google-research/tree/master/tsmixer

研究动机与目标

• 评估跨变量信息是否有助于在常见基准上进行时间序列预测。
• 开发一个轻量、可扩展的全MLP架构，捕捉时间模式和跨变量交互。
• 探索将辅助信息（静态特征和未来时间特征）合并的扩展。
• 在长期预测数据集上与最先进的多变量和单变量模型进行比较。

提出的方法

• 引入 Time-Series Mixer (TSMixer)，其具有交错的时间混合和特征混合 MLP。
• 时间混合 MLP 在特征之间共享以建模时间模式；特征混合 MLP 在时间步之间共享以利用协变量信息。
• 使用时间投影将输入长度 L 映射到预测长度 T，并实现时间学习。
• 引入残差连接以及跨时间和特征维度的二维归一化以实现稳定训练。
• 提供一个 TMix-Only 变体，仅进行时间混合，以单独隔离跨变量混合的影响。
• 通过将输入对齐成共同形状并对拼接后的特征应用混合，将 TSMixer 扩展到辅助信息（静态特征和未来时间变化特征）。

Figure 1: TSMixer for multivariate time series forecasting. The columns of the inputs means different features/variates and the rows are time steps. The fully-connected operations are row-wise. TSMixer contains interleaving time-mixing and feature-mixing MLPs to aggregate information. The number of

实验结果

研究问题

• RQ1 跨变量信息是否为时间序列预测在标准基准上提供可靠的收益？
• RQ2 一种全MLP混合架构是否能在长期预测中与单变量模型相匹配并超过基于Transformer的多变量模型？
• RQ3 TSMixer 如何随更长的回顾窗口和大规模数据集扩展？
• RQ4 TSMixer 能否有效利用诸如静态特征和未来时间特征等辅助信息？

主要发现

• TSMixer 在最先进的单变量模型上具有竞争力，并在常见的长期预测基准上显著超过其他多变量模型。
• 在大规模的 M5 零售数据上，跨变量信息带来显著改进，TSMixer 能有效利用辅助信息。
• TMix-Only（仅时间混合）与 PatchTST 相当，表明在这些基准上并非总是需要跨变量混合。
• TSMixer 能比多变量 Transformer 基模型更好地利用更长的回看窗口，表现出更好的泛化和对长序列的过拟合降低。
• 扩展到辅助信息（静态和未来特征）带来强劲的性能，在 M5 场景下常常超过如 DeepAR 和 TFT 等工业基线。

Figure 3: The architecture of TMix-Only. It is similar to TSMixer but only applies time-mixing.