[论文解读] The Capacity and Robustness Trade-off: Revisiting the Channel Independent Strategy for Multivariate Time Series Forecasting

Lu Han, Han-Jia Ye+1 more|arXiv (Cornell University)|Apr 11, 2023

Forecasting Techniques and Applications被引用 11

一句话总结

本文表明在多变量时间序列预测中，通道独立（CI）训练通常优于通道相关（CD）训练，分析容量-鲁棒性权衡，并提出带正则化的预测残差（PRReg）以提升 CD。

ABSTRACT

Multivariate time series data comprises various channels of variables. The multivariate forecasting models need to capture the relationship between the channels to accurately predict future values. However, recently, there has been an emergence of methods that employ the Channel Independent (CI) strategy. These methods view multivariate time series data as separate univariate time series and disregard the correlation between channels. Surprisingly, our empirical results have shown that models trained with the CI strategy outperform those trained with the Channel Dependent (CD) strategy, usually by a significant margin. Nevertheless, the reasons behind this phenomenon have not yet been thoroughly explored in the literature. This paper provides comprehensive empirical and theoretical analyses of the characteristics of multivariate time series datasets and the CI/CD strategy. Our results conclude that the CD approach has higher capacity but often lacks robustness to accurately predict distributionally drifted time series. In contrast, the CI approach trades capacity for robust prediction. Practical measures inspired by these analyses are proposed to address the capacity and robustness dilemma, including a modified CD method called Predict Residuals with Regularization (PRReg) that can surpass the CI strategy. We hope our findings can raise awareness among researchers about the characteristics of multivariate time series and inspire the construction of better forecasting models.

研究动机与目标

• 研究为什么 CI（将多变量序列视为独立的单变量序列）在多种数据集和模型中可能优于 CD（将所有通道一起使用）
• 描述 CI 与 CD 策略在多变量时间序列预测中的容量与鲁棒性权衡
• 提供实际策略以提升预测性能，包括用于改进基于 CD 的模型的新目标（PRReg）

提出的方法

• 形式化多变量时间序列预测中的 CD 与 CI 训练策略。
• 在九个真实数据集上使用非深度模型和深度模型进行广泛的经验比较。
• 通过自相关函数（ACF）分析分布漂移及其对 CD 与 CI 的影响。
• 通过线性自回归（AR）视角和 Yule-Walker 方程提供理论联系。
• 提出 Predict Residuals with Regularization（PRReg）以解决 CD 的非鲁棒性。

实验结果

研究问题

• RQ1 通道独立（CI）策略是否在不同数据集和模型上始终优于通道相关（CD）？
• RQ2 在容量与鲁棒性方面，特别是在分布漂移下，CI 的经验优势如何解释？
• RQ3 我们如何修改训练目标以改进基于 CD 的模型并有可能超越 CI？
• RQ4 哪些实际因素影响 CD/CI 的性能，我们如何利用它们来设计更好的预测模型？

主要发现

• CI 在大多数实验中优于 CD，且差异通常相当显著；在复杂模型上 CI 的改进平均约20%及以上。
• CI 不仅降低误差，还降低性能方差，表明结果比 CD 更稳健和一致。
• CD 具有更高的容量但对分布漂移的鲁棒性较低，而 CI 容量较低但鲁棒性更高。
• 作者在经验和线性情形中的理论论证表明，CI 依赖于跨通道的平均 ACF，漂移程度小于单个通道的 ACF，从而提高鲁棒性。
• 一个新目标，Predict Residuals with Regularization（PRReg），在许多情况下可以超越 CI，解决 CD 的非鲁棒性。
• 该工作建议将训练策略与算法设计分离，以便进行公平比较和进一步改进。