[论文解读] Bi-Mamba+: Bidirectional Mamba for Time Series Forecasting

Aobo Liang, Xingguo Jiang+3 more|arXiv (Cornell University)|Apr 24, 2024

Stock Market Forecasting Methods被引用 11

一句话总结

Bi-Mamba4TS 提出一个双向 Mamba 编码器，带有一个系列关系感知（SRA）决策器，自动在通道无关与通道混合标记之间进行选择，通过修补来丰富局部信息，用于长期多变量时间序列预测。

ABSTRACT

Long-term time series forecasting (LTSF) provides longer insights into future trends and patterns. Over the past few years, deep learning models especially Transformers have achieved advanced performance in LTSF tasks. However, LTSF faces inherent challenges such as long-term dependencies capturing and sparse semantic characteristics. Recently, a new state space model (SSM) named Mamba is proposed. With the selective capability on input data and the hardware-aware parallel computing algorithm, Mamba has shown great potential in balancing predicting performance and computational efficiency compared to Transformers. To enhance Mamba's ability to preserve historical information in a longer range, we design a novel Mamba+ block by adding a forget gate inside Mamba to selectively combine the new features with the historical features in a complementary manner. Furthermore, we apply Mamba+ both forward and backward and propose Bi-Mamba+, aiming to promote the model's ability to capture interactions among time series elements. Additionally, multivariate time series data in different scenarios may exhibit varying emphasis on intra- or inter-series dependencies. Therefore, we propose a series-relation-aware decider that controls the utilization of channel-independent or channel-mixing tokenization strategy for specific datasets. Extensive experiments on 8 real-world datasets show that our model achieves more accurate predictions compared with state-of-the-art methods.

研究动机与目标

• Motivate long-term multivariate time series forecasting (LTSF) and address Transformer efficiency/accuracy trade-offs.
• Leverage State Space Models (SSM) with Mamba for linear-time complexity and long-range dependency modeling.
• Automate tokenization strategy selection via a Series-Relation-Aware (SRA) decider based on Pearson correlations.
• Enrich time series representations by patching to capture finer-grained evolutionary patterns.
• Demonstrate superior forecasting performance across seven real-world datasets compared to state-of-the-art baselines.

提出的方法

• Divide input time series into patches to obtain patch-wise tokens.
• Use a Series-Relation-Aware (SRA) decider to automatically choose between channel-independent and channel-mixing tokenization based on Pearson correlations.
• Apply a bidirectional Mamba encoder with forward and backward branches to model intra- and inter-series dependencies.
• Employ RevIN to normalize non-stationary inputs and denormalize outputs.
• Train with MSE loss and use ADAM optimization with early stopping across multiple encoder layers.

实验结果

研究问题

• RQ1 Can a data-driven decider effectively select between channel-independent and channel-mixing tokenization to suit different MTS datasets?
• RQ2 Does patch-based tokenization combined with bidirectional Mamba improve long-term forecasting accuracy over Transformer-based and other SSM-based models?
• RQ3 What is the impact of bidirectional design on modeling intra- vs inter-series dependencies in MTS data?
• RQ4 How robust is Bi-Mamba4TS to hyper-parameter variations across diverse real-world datasets?

主要发现

• Bi-Mamba4TS achieves superior forecasting accuracy across seven real-world datasets for multiple horizons compared with strong baselines.
• The model reduces MSE/MAE versus iTransformer by average margins of 4.92% and 2.16%, respectively.
• Using channel-mixing on Weather, Traffic, Electricity and channel-independent on four ETT datasets shows dataset-dependent gains.
• Ablation studies indicate the SRA decider and bidirectional Mamba contribute meaningfully to performance gains.
• Removing the SRA decider or the backward Mamba branch degrades performance, evidencing the importance of strategy selection and bidirectionality.