[论文解读] CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuning

Peiyuan Liu, Hang Guo+6 more|arXiv (Cornell University)|Mar 12, 2024

Stock Market Forecasting Methods被引用 5

一句话总结

CALF 引入一种跨模态微调框架，通过静态和动态知识蒸馏将时间序列与预训练的 LLM 嵌入对齐，在长期与短期预测中达到 state-of-the-art，并提升泛化能力。

ABSTRACT

Deep learning (e.g., Transformer) has been widely and successfully used in multivariate time series forecasting (MTSF). Unlike existing methods that focus on training models from a single modal of time series input, large language models (LLMs) based MTSF methods with cross-modal text and time series input have recently shown great superiority, especially with limited temporal data. However, current LLM-based MTSF methods usually focus on adapting and fine-tuning LLMs, while neglecting the distribution discrepancy between textual and temporal input tokens, thus leading to sub-optimal performance. To address this issue, we propose a novel Cross-Modal LLM Fine-Tuning (CALF) framework for MTSF by reducing the distribution discrepancy between textual and temporal data, which mainly consists of the temporal target branch with temporal input and the textual source branch with aligned textual input. To reduce the distribution discrepancy, we develop the cross-modal match module to first align cross-modal input distributions. Additionally, to minimize the modality distribution gap in both feature and output spaces, feature regularization loss is developed to align the intermediate features between the two branches for better weight updates, while output consistency loss is introduced to allow the output representations of both branches to correspond effectively. Thanks to the modality alignment, CALF establishes state-of-the-art performance for both long-term and short-term forecasting tasks with low computational complexity, and exhibiting favorable few-shot and zero-shot abilities similar to that in LLMs. Code is available at https://github.com/Hank0626/LLaTA.

研究动机与目标

• 推动缩小时间序列数据与文本型 LLM 表征之间的模态差距，以提升预测泛化能力。
• 利用跨模态知识蒸馏将静态（嵌入空间）和动态（上下文）LLM 知识迁移到时序预测模型。
• 设计一个两分支架构，将时间数据处理与基于 LLM 的文本知识整合。
• 开发训练损失和效率策略，以在避免灾难性遗忘的情况下实现有效迁移。

提出的方法

• 两分支框架：一个文本模态分支（基于 LLM），以及一个时序模态分支（时间序列处理。

实验结果

研究问题

• RQ1 Can cross-modal knowledge distillation reduce the modality gap between time series and text embeddings to improve forecasting performance?
• RQ2 How can static (embedding-based) and dynamic (contextual) knowledge from pre-trained LLMs be leveraged to enhance time series forecasting and generalization?
• RQ3 What are effective loss designs to enforce cross-modal consistency and prevent forgetting during fine-tuning?
• RQ4 Does the proposed LLaTA framework achieve state-of-the-art performance across long-term, short-term, and few/zero-shot forecasting scenarios?

主要发现

• LLaTA achieves state-of-the-art performance on both long- and short-term forecasting across multiple real-world datasets.
• In long-term forecasting, LLaTA delivers top results in the majority of evaluations and consistently outperforms strong baselines including PatchTST and GPT4TS.
• In short-term forecasting on M4 datasets, LLaTA attains best results in 14 of 15 categories and surpasses leading methods.
• LLaTA demonstrates strong few-shot and zero-shot generalization, outperforming baselines in cross-domain transfer scenarios.
• Ablation studies confirm that combining supervised, feature regularization, and modal consistency losses yields the best MSE/MAE performance on tested datasets.