[Paper Review] Frequency Adaptive Normalization For Non-stationary Time Series Forecasting

Weiwei Ye, Songgaojun Deng+2 more|arXiv (Cornell University)|Sep 30, 2024

Stock Market Forecasting MethodsDecision Sciences3 citations

TL;DR

This paper proposes Frequency Adaptive Normalization (FAN), a model-agnostic normalization method that uses instance-wise Fourier transforms to identify and remove dynamic trend and seasonal components in non-stationary time series. By modeling the evolution of dominant frequency components via a simple MLP, FAN improves forecasting performance across four backbone models on eight benchmark datasets, achieving 7.76% to 37.90% average MSE reduction.

ABSTRACT

Time series forecasting typically needs to address non-stationary data with evolving trend and seasonal patterns. To address the non-stationarity, reversible instance normalization has been recently proposed to alleviate impacts from the trend with certain statistical measures, e.g., mean and variance. Although they demonstrate improved predictive accuracy, they are limited to expressing basic trends and are incapable of handling seasonal patterns. To address this limitation, this paper proposes a new instance normalization solution, called frequency adaptive normalization (FAN), which extends instance normalization in handling both dynamic trend and seasonal patterns. Specifically, we employ the Fourier transform to identify instance-wise predominant frequent components that cover most non-stationary factors. Furthermore, the discrepancy of those frequency components between inputs and outputs is explicitly modeled as a prediction task with a simple MLP model. FAN is a model-agnostic method that can be applied to arbitrary predictive backbones. We instantiate FAN on four widely used forecasting models as the backbone and evaluate their prediction performance improvements on eight benchmark datasets. FAN demonstrates significant performance advancement, achieving 7.76% ~ 37.90% average improvements in MSE.

Motivation & Objective

• To address the limitation of existing reversible instance normalization methods that rely only on time-domain statistics (e.g., mean, variance) and fail to capture evolving seasonal patterns.
• To develop a normalization technique that can effectively model both non-stationary trends and seasonal components in time series data.
• To enable model-agnostic integration into existing forecasting backbones for performance improvement.
• To explicitly model the evolution of non-stationary patterns from input to output, rather than assuming they remain constant.

Proposed method

• FAN applies instance-wise Fourier transforms to extract the top K dominant frequency components from each input sequence, representing the primary non-stationary factors.
• It normalizes the input by removing these dominant frequency components, thereby reducing non-stationarity in the input data.
• A pattern adaptation module, implemented as a simple MLP, predicts the evolution of the top K frequency components from input to output sequence.
• The predicted frequency components are then used to reconstruct the output, restoring the non-stationary structure in a differentiable and learnable manner.
• The method is designed to be model-agnostic, allowing integration with any deep learning backbone for time series forecasting.
• The approach explicitly models the discrepancy between input and output frequency components, enabling better generalization to non-stationary dynamics.

Experimental results

Research questions

• RQ1 Can frequency-domain representation via Fourier transform better capture non-stationary components such as evolving trends and seasonal patterns than time-domain statistics?
• RQ2 How effective is a learnable frequency adaptation module in modeling the evolution of non-stationary patterns across time steps?
• RQ3 To what extent can a model-agnostic normalization method like FAN improve performance across diverse forecasting backbones and datasets?
• RQ4 Does FAN outperform existing normalization techniques that rely solely on mean and variance in handling complex non-stationary time series?

Key findings

• FAN achieves average MSE improvements ranging from 7.76% to 37.90% across eight benchmark datasets when applied to four different forecasting backbones.
• The method significantly outperforms existing reversible instance normalization techniques that rely only on time-domain statistics such as mean and variance.
• The pattern adaptation module, using a simple MLP, effectively captures the evolution of dominant frequency components from input to output, improving predictive accuracy.
• FAN demonstrates consistent performance gains across diverse domains, including traffic, electricity, exchange rates, and weather forecasting.
• The ablation study confirms that frequency-based normalization is more effective than time-domain normalization for capturing non-stationary dynamics.
• The method is robust to varying sequence lengths and non-stationary patterns, including signals with linearly varying frequency, which are poorly handled by prior approaches.