[Paper Review] Rethinking Channel Dependence for Multivariate Time Series Forecasting: Learning from Leading Indicators

Lifan Zhao, Yanyan Shen|arXiv (Cornell University)|Jan 31, 2024

Forecasting Techniques and Applications4 citations

TL;DR

This paper proposes LIFT, a plug-and-play method for multivariate time series forecasting that leverages dynamically identified, locally stationary lead-lag relationships between variates. By estimating leading indicators and their time lags at each step and refining forecasts using a frequency-domain aware refiner, LIFT improves state-of-the-art models by 5.5% on average across six real-world datasets.

ABSTRACT

Recently, channel-independent methods have achieved state-of-the-art performance in multivariate time series (MTS) forecasting. Despite reducing overfitting risks, these methods miss potential opportunities in utilizing channel dependence for accurate predictions. We argue that there exist locally stationary lead-lag relationships between variates, i.e., some lagged variates may follow the leading indicators within a short time period. Exploiting such channel dependence is beneficial since leading indicators offer advance information that can be used to reduce the forecasting difficulty of the lagged variates. In this paper, we propose a new method named LIFT that first efficiently estimates leading indicators and their leading steps at each time step and then judiciously allows the lagged variates to utilize the advance information from leading indicators. LIFT plays as a plugin that can be seamlessly collaborated with arbitrary time series forecasting methods. Extensive experiments on six real-world datasets demonstrate that LIFT improves the state-of-the-art methods by 5.5% in average forecasting performance. Our code is available at https://github.com/SJTU-Quant/LIFT.

Motivation & Objective

• To address the limitation of channel-independent methods in missing useful cross-channel dependencies in multivariate time series (MTS).
• To investigate whether locally stationary lead-lag relationships between variates can be exploited to improve forecasting accuracy.
• To develop a method that dynamically identifies leading indicators and their time lags for real-time forecasting use.
• To create a plug-and-play module compatible with any backbone model, enhancing performance without architectural overhaul.

Proposed method

• Proposes an efficient cross-correlation algorithm to dynamically estimate leading indicators and their leading steps at each time step.
• Introduces a target-oriented shift trick to align each variate with its leading indicators based on estimated lead times.
• Employs a backbone model (e.g., DLinear, LightMTS) for initial forecasting, followed by a Lead-aware Refiner that filters and fuses informative frequency components from leading indicators.
• Designs an Adaptive Frequency Mixer within the Lead-aware Refiner to selectively absorb desirable frequency components from leading indicators while suppressing noise.
• Uses a variate state representation to model context-dependent lead-lag patterns, especially in heterogeneous MTS with diverse sensor measurements.
• Integrates the entire pipeline as a plug-in module, enabling seamless use with any existing MTS forecasting model.

Experimental results

Research questions

• RQ1 Can locally stationary lead-lag relationships between variates be effectively exploited to improve multivariate time series forecasting?
• RQ2 How can leading indicators and their time lags be dynamically estimated in real time without prior assumptions?
• RQ3 What is the impact of incorporating frequency-domain signals from leading indicators on forecasting accuracy?
• RQ4 How does the performance of LIFT vary under distribution shifts in lead-lag relationships across different datasets?
• RQ5 Can a lightweight, channel-independent baseline like LightMTS outperform complex models when enhanced with lead-lag modeling?

Key findings

• LIFT improves the state-of-the-art performance by an average of 5.5% across six real-world datasets, demonstrating consistent gains over existing methods.
• The LightMTS baseline, when enhanced with LIFT, outperforms complex models like PatchTST by 3.2% on the Weather dataset and 0.7% on Electricity.
• The ablation study confirms that both the influence and difference components in the Lead-aware Refiner are essential, with removal of either reducing performance significantly.
• The variant without filtering performs worst, indicating that adaptive noise suppression in frequency components is critical for robustness.
• Optimal performance is achieved with moderate values of K (number of leading indicators) and N (number of variate states), with over-estimation leading to performance degradation on sparse or noisy datasets.
• The variate state representation is especially crucial for heterogeneous MTS like Weather, where ignoring it causes the largest performance drop.