[Paper Review] FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting

Tian Zhou, Ziqing Ma+6 more|arXiv (Cornell University)|May 18, 2022

Stock Market Forecasting MethodsDecision Sciences103 citations

TL;DR

FiLM introduces a Frequency Improved Legendre Memory Model that combines Legendre projections with Fourier-based noise reduction and low-rank approximations to robustly preserve history for long-term forecasting, achieving state-of-the-art results on multiple datasets.

ABSTRACT

Recent studies have shown that deep learning models such as RNNs and Transformers have brought significant performance gains for long-term forecasting of time series because they effectively utilize historical information. We found, however, that there is still great room for improvement in how to preserve historical information in neural networks while avoiding overfitting to noise presented in the history. Addressing this allows better utilization of the capabilities of deep learning models. To this end, we design a extbf{F}requency extbf{i}mproved extbf{L}egendre extbf{M}emory model, or {\bf FiLM}: it applies Legendre Polynomials projections to approximate historical information, uses Fourier projection to remove noise, and adds a low-rank approximation to speed up computation. Our empirical studies show that the proposed FiLM significantly improves the accuracy of state-of-the-art models in multivariate and univariate long-term forecasting by ( extbf{20.3\%}, extbf{22.6\%}), respectively. We also demonstrate that the representation module developed in this work can be used as a general plug-in to improve the long-term prediction performance of other deep learning modules. Code is available at https://github.com/tianzhou2011/FiLM/

Motivation & Objective

• Motivate robust long-term time series forecasting by preserving useful historical information while suppressing noise.
• Develop a representation framework based on Legendre projections enhanced with frequency-domain filtering to improve forecasting accuracy.
• Create a modular architecture (LPU and FEL) that can serve as a plug-in to improve various backbones for long-term forecasting.
• Demonstrate that FiLM achieves superior accuracy and efficiency across diverse real-world datasets, and analyze the impact of its components.

Proposed method

• Use Legendre Polynomial projections to compress historical information via a Legendre Projection Unit (LPU) that updates a fixed-size memory.
• Incorporate a Frequency Enhanced Layer (FEL) that reduces dimensionality and noise through a combination of Fourier analysis and low-rank approximation.
• Leverage a mixture of multiscale experts to capture information at different time horizons (T, 2T, ..., nT).
• Optionally apply RevIN data normalization to improve robustness, recognizing it may slow training.
• Provide theoretical results (Theorems 1–3) linking Legendre projection accuracy, noise accumulation, and the efficacy of partial Fourier-based coefficient sampling.

Experimental results

Research questions

• RQ1 How can historical information be preserved for long-term forecasting without overfitting to noisy past data?
• RQ2 Can a frequency-domain layer effectively denoise Legendre-based representations to improve forecasting accuracy?
• RQ3 Does a multiscale mixture of experts improve long-horizon predictions across domains?
• RQ4 Can FiLM serve as a plug-in to improve existing forecasting backbones with Legendre-FEL representations?

Key findings

• FiLM significantly improves forecast accuracy over state-of-the-art models on six benchmark datasets for both multivariate and univariate long-term forecasting.
• Across experiments, FiLM achieves up to 20.3% relative MSE reduction in multivariate forecasting and up to 22.6% relative MSE reduction in univariate forecasting compared to FEDformer.
• A low-rank approximation in the FEL reduces learnable parameters to as little as 0.1%–0.4% with minor accuracy loss, enabling substantial memory savings.
• The LPU layer provides a boosting effect across backbones (MLP, LSTM, CNN, Attention), often outperforming comparable linear baselines.
• Normal multiscale structure consistently improves performance, while RevIN is optional and dataset-dependent in effectiveness.