[Paper Review] SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction

Minhao Liu, Ailing Zeng+5 more|arXiv (Cornell University)|Jun 17, 2021

Time Series Analysis and Forecasting56 references245 citations

TL;DR

SCINet introduces a hierarchical downsample-convolve-interact architecture (SCI-Block) to model multiresolution temporal dynamics for time series forecasting, achieving strong performance across diverse datasets compared to RNN, TCN, and Transformer baselines.

ABSTRACT

One unique property of time series is that the temporal relations are largely preserved after downsampling into two sub-sequences. By taking advantage of this property, we propose a novel neural network architecture that conducts sample convolution and interaction for temporal modeling and forecasting, named SCINet. Specifically, SCINet is a recursive downsample-convolve-interact architecture. In each layer, we use multiple convolutional filters to extract distinct yet valuable temporal features from the downsampled sub-sequences or features. By combining these rich features aggregated from multiple resolutions, SCINet effectively models time series with complex temporal dynamics. Experimental results show that SCINet achieves significant forecasting accuracy improvements over both existing convolutional models and Transformer-based solutions across various real-world time series forecasting datasets. Our codes and data are available at https://github.com/cure-lab/SCINet.

Motivation & Objective

• Motivate time series forecasting by leveraging the property that temporal relations are largely preserved after downsampling into two sub-sequences.
• Propose a novel encoder-decoder architecture (SCINet) with SCI-Blocks to extract multiresolution temporal features.
• Show that hierarchical downsample-convolve-interact operations improve forecasting accuracy across real-world datasets.
• Analyse the learned representations using permutation entropy to assess predictability improvements.

Proposed method

• Introduce SCI-Block that splits input into even/odd sub-sequences, applies distinct convolutions, and uses interactive learning to exchange information between sub-sequences via affine transformations.
• Construct SCINet as a binary-tree of SCI-Blocks to capture multi-resolution temporal dependencies and provide local and global views of the series.
• Optionally stack multiple SCINets with intermediate supervision (Stacked SCINet) to further enhance forecast performance.
• Use a residual connection to add the enhanced representation to the original series and decode via a simple fully-connected network.
• Train the stacked model with multi-stage L1 losses, summing losses across all SCINets.

Experimental results

Research questions

• RQ1 Can downsample-convolve-interact blocks capture complex temporal dynamics more effectively than standard dilated convolutions?
• RQ2 Do SCINet variants (single, stacked, with intermediate supervision) outperform RNN-, TCN-, and Transformer-based time series forecasting models across diverse datasets?
• RQ3 How does SCINet perform on short-term, long-term, and spatial-temporal forecasting tasks without explicit spatial modeling?
• RQ4 Is the learned representation demonstrably more predictable, as measured by permutation entropy, than the original input?

Key findings

• SCINet consistently outperforms existing TSF models (including RNN/TCN/Transformer baselines) across a wide range of real-world datasets.
• Long-term forecasting results show SCINet achieving state-of-the-art performance in many settings, with substantial average improvements on selected benchmarks.
• In spatial-temporal tasks (e.g., traffic data), SCINet provides competitive forecasting accuracy without explicit spatial relation modeling.
• Permutation entropy analysis indicates the enhanced representations learned by SCINet are less complex and more predictable than the original inputs.
• The architecture achieves favorable computational complexity (worst-case O(T log T)) comparable to TCNs, with potentially smaller required depth (L ≤ 5 in many cases).