[Paper Review] Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structures

Tianping Zhang, Yizhuo Zhang+5 more|arXiv (Cornell University)|Jul 4, 2022

Time Series Analysis and Forecasting88 citations

TL;DR

LightTS presents a lightweight MLP-based model with continuous and interval sampling to capture short- and long-term patterns in multivariate time series, achieving state-of-the-art or competitive results with far fewer FLOPS and stable robustness.

ABSTRACT

Multivariate time series forecasting has seen widely ranging applications in various domains, including finance, traffic, energy, and healthcare. To capture the sophisticated temporal patterns, plenty of research studies designed complex neural network architectures based on many variants of RNNs, GNNs, and Transformers. However, complex models are often computationally expensive and thus face a severe challenge in training and inference efficiency when applied to large-scale real-world datasets. In this paper, we introduce LightTS, a light deep learning architecture merely based on simple MLP-based structures. The key idea of LightTS is to apply an MLP-based structure on top of two delicate down-sampling strategies, including interval sampling and continuous sampling, inspired by a crucial fact that down-sampling time series often preserves the majority of its information. We conduct extensive experiments on eight widely used benchmark datasets. Compared with the existing state-of-the-art methods, LightTS demonstrates better performance on five of them and comparable performance on the rest. Moreover, LightTS is highly efficient. It uses less than 5% FLOPS compared with previous SOTA methods on the largest benchmark dataset. In addition, LightTS is robust and has a much smaller variance in forecasting accuracy than previous SOTA methods in long sequence forecasting tasks.

Motivation & Objective

• Motivate whether simple, lightweight models can rival complex architectures for multivariate time series forecasting.
• Introduce two down-sampling strategies that preserve information while reducing input length.
• Develop an MLP-based architecture that exchanges information across down-sampled subsequences and variables.
• Demonstrate LightTS’s efficiency, robustness, and forecasting accuracy on multiple benchmarks.

Proposed method

• Apply continuous sampling and interval sampling to split each time series into non-overlapping subsequences without discarding tokens.
• Use Information Exchange Blocks (IEBlocks) with a bottleneck design to perform temporal and channel projections for feature extraction and inter-series interaction.
• Combine features from both sampling paths and across variables to form a final prediction via a third IEBlock.
• Train end-to-end with a lightweight architecture that processes long sequences efficiently.

Experimental results

Research questions

• RQ1 Can a pure MLP-based model with light down-sampling match or exceed state-of-the-art methods on multivariate time series forecasting?
• RQ2 Do continuous and interval sampling enable effective capture of short-term and long-term patterns while improving efficiency?
• RQ3 Is LightTS robust and scalable to long input sequences across diverse datasets?

Key findings

• LightTS achieves state-of-the-art or competitive results on eight benchmark datasets, outperforming several baselines in long sequence forecasting.
• LightTS requires less than 5% of FLOPS of previous SOTA methods on the largest benchmark dataset.
• LightTS demonstrates significantly faster training per epoch than strong baselines in multiple settings.
• LightTS shows lower variance in forecasting accuracy across seeds, indicating improved robustness for long sequence tasks.