[Paper Review] VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasters

Mouxiang Chen, Lefei Shen+4 more|arXiv (Cornell University)|Aug 30, 2024

Forecasting Techniques and ApplicationsDecision Sciences3 citations

TL;DR

VisionTS proposes a novel zero-shot time series forecasting framework that repurposes vision-based masked autoencoders pre-trained on ImageNet to forecast time series without any fine-tuning. By transforming 1D time series into 2D image-like matrices and leveraging the self-supervised representation of visual models, VisionTS achieves state-of-the-art zero-shot performance and surpasses existing foundation models like Moirai Large, demonstrating that visual models can serve as a 'free lunch' for time series forecasting.

ABSTRACT

Foundation models have emerged as a promising approach in time series forecasting (TSF). Existing approaches either repurpose large language models (LLMs) or build large-scale time series datasets to develop TSF foundation models for universal forecasting. However, these methods face challenges due to the severe cross-domain gap or in-domain heterogeneity. This paper explores a new road to building a TSF foundation model from rich, high-quality natural images. Our key insight is that a visual masked autoencoder, pre-trained on the ImageNet dataset, can naturally be a numeric series forecaster. By reformulating TSF as an image reconstruction task, we bridge the gap between image pre-training and TSF downstream tasks. Surprisingly, without further adaptation in the time series domain, the proposed VisionTS could achieve better zero-shot forecast performance than existing TSF foundation models. With fine-tuning for one epoch, VisionTS could further improve the forecasting and achieve state-of-the-art performance in most cases. Extensive experiments reveal intrinsic similarities between images and real-world time series, suggesting that visual models may offer a "free lunch" for TSF and highlight the potential for future cross-modality research. Our code is publicly available at https://github.com/Keytoyze/VisionTS.

Motivation & Objective

• To explore whether pre-trained visual models can serve as foundation models for zero-shot time series forecasting.
• To address the limitations of existing text-based and time-series-based foundation models, which face cross-domain gaps or data heterogeneity issues.
• To bridge the modality gap between images and time series by exploiting their shared structural and physical characteristics.
• To evaluate whether visual masked autoencoders pre-trained on ImageNet can generalize to time series forecasting without further adaptation.
• To demonstrate that visual models can be a low-cost, high-performance alternative to domain-specific foundation models in time series forecasting.

Proposed method

• Transform 1D time series data into 2D matrices through segmentation to preserve temporal patterns.
• Reformulate time series forecasting as a patch-level image reconstruction task using visual masked autoencoders (MAE).
• Utilize a vision MAE pre-trained on ImageNet to leverage learned representations of common time series features like trend, seasonality, and stationarity.
• Apply masking to future forecasting windows during inference, treating them as missing patches to be reconstructed.
• Perform zero-shot forecasting by directly using the pre-trained visual model without any fine-tuning on time series data.
• Fine-tune only the layer normalization components for minimal adaptation, achieving state-of-the-art performance on most benchmarks.

Experimental results

Research questions

• RQ1 Can a vision-based masked autoencoder pre-trained on ImageNet achieve competitive zero-shot performance in time series forecasting?
• RQ2 How does the performance of VisionTS compare to existing text-based and time-series-based foundation models in zero-shot settings?
• RQ3 What is the impact of different fine-tuning strategies on VisionTS's forecasting accuracy?
• RQ4 Does the intrinsic similarity between images and time series enable effective transfer learning from vision to time series forecasting?
• RQ5 In which forecasting scenarios does VisionTS outperform or underperform relative to strong baselines like Moirai Large?

Key findings

• VisionTS achieves state-of-the-art zero-shot forecasting performance on multiple benchmarks, outperforming Moirai Large, the largest existing foundation model, without any fine-tuning.
• On the ETTh1 dataset, VisionTS achieves an MAE of 0.395 in the zero-shot setting, compared to Moirai Large’s 0.534, demonstrating a 25.8% relative improvement.
• With only one epoch of fine-tuning, VisionTS achieves SOTA results across most long-term forecasting benchmarks, with average MAE improvements of up to 15% over zero-shot performance.
• Abllation studies confirm that visual knowledge is crucial: removing the visual model (w/o VM) degrades performance to 0.534 MAE on ETTh1, while the full VisionTS model achieves 0.395.
• Fine-tuning only the layer normalization (LN) yields the best performance, with an average MAE of 0.333 on the benchmark, outperforming full fine-tuning and other parameter-specific strategies.
• Visualization shows that VisionTS generalizes well to regular patterns (e.g., Fig. 8), but can overfit trends in low-pattern inputs, where Moirai Large performs better (Fig. 11).