[Paper Review] Frequency-domain MLPs are More Effective Learners in Time Series Forecasting

Kun Yi, Qi Zhang+8 more|arXiv (Cornell University)|Nov 10, 2023

Stock Market Forecasting Methods91 citations

TL;DR

FreTS introduces frequency-domain MLPs with two learners (channel and temporal) to learn global spectral dependencies for time series forecasting, achieving superior results across 13 real-world benchmarks versus state-of-the-art baselines.

ABSTRACT

Time series forecasting has played the key role in different industrial, including finance, traffic, energy, and healthcare domains. While existing literatures have designed many sophisticated architectures based on RNNs, GNNs, or Transformers, another kind of approaches based on multi-layer perceptrons (MLPs) are proposed with simple structure, low complexity, and {superior performance}. However, most MLP-based forecasting methods suffer from the point-wise mappings and information bottleneck, which largely hinders the forecasting performance. To overcome this problem, we explore a novel direction of applying MLPs in the frequency domain for time series forecasting. We investigate the learned patterns of frequency-domain MLPs and discover their two inherent characteristic benefiting forecasting, (i) global view: frequency spectrum makes MLPs own a complete view for signals and learn global dependencies more easily, and (ii) energy compaction: frequency-domain MLPs concentrate on smaller key part of frequency components with compact signal energy. Then, we propose FreTS, a simple yet effective architecture built upon Frequency-domain MLPs for Time Series forecasting. FreTS mainly involves two stages, (i) Domain Conversion, that transforms time-domain signals into complex numbers of frequency domain; (ii) Frequency Learning, that performs our redesigned MLPs for the learning of real and imaginary part of frequency components. The above stages operated on both inter-series and intra-series scales further contribute to channel-wise and time-wise dependency learning. Extensive experiments on 13 real-world benchmarks (including 7 benchmarks for short-term forecasting and 6 benchmarks for long-term forecasting) demonstrate our consistent superiority over state-of-the-art methods.

Motivation & Objective

• Motivate time series forecasting with simple, low-complexity MLPs while addressing the limitations of point-wise mappings and information bottlenecks.
• Propose a frequency-domain learning framework (FreTS) that leverages global spectral views and energy compaction to capture dependencies.
• Design two frequency learners (channel and temporal) operating on transformed frequency components to model inter-series and intra-series dynamics.
• Demonstrate effectiveness and efficiency of FreTS on 13 real-world datasets, comparing against state-of-the-art methods.
• Provide ablation and compatibility analyses to validate the contributions of FreMLP and the two frequency learners.

Proposed method

• Transform time-domain inputs to frequency domain via Discrete Fourier Transform (DFT) to obtain real/imaginary components.
• Apply two frequency learners: Frequency Channel Learner (handles inter-channel dependencies) and Frequency Temporal Learner (handles temporal dependencies) operating on frequency representations.
• Use FreMLP (complex-valued MLPs) to learn mappings separately on real and imaginary parts and combine them to form frequency-domain outputs.
• Perform Domain Conversion and Inversion along both channel and time dimensions to enable spectral learning and subsequent reconstruction to the time domain.
• Extend inputs with a learnable dimension-extension block to enrich representations before frequency-domain processing.
• Conclude with a two-layer projection (FFN) to generate final horizon forecasts, avoiding error accumulation.

(a) Left : time domain. Right : frequency domain.

Experimental results

Research questions

• RQ1 How does FreTS perform compared with state-of-the-art forecasting models across short-term and long-term benchmarks?
• RQ2 What is the contribution of the frequency channel versus the frequency temporal learner to FreTS's performance?
• RQ3 Does FreMLP provide measurable gains when embedded into existing MLP-based forecasting models?
• RQ4 What are the efficiency characteristics of FreTS compared to Transformer- and GNN-based baselines?

Key findings

• FreTS consistently outperforms baselines on both short-term and long-term forecasting across 13 real-world datasets.
• In ablations, the frequency channel learner substantially benefits short-term forecasting, while the frequency temporal learner shows strong effectiveness for long-term forecasting.
• Replacing conventional MLPs with FreMLP in DLinear and NLinear yields performance improvements across datasets (e.g., Exchange and Weather).
• FreTS demonstrates favorable efficiency, with complexity O(N log N + L log L) and reduced parameter counts and training time relative to several Transformer- and GNN-based methods.
• The two-stage domain approach (domain conversion and inversion) across channel and time dimensions enables learning of global spectral dependencies and energy-focused representations, contributing to improved forecasting.

(b) Left : time domain. Right : frequency domain.