[Paper Review] High-dimensional Time Series Prediction with Missing Values

Hsiang‐Fu Yu, Nikhil Rao+1 more|arXiv (Cornell University)|Sep 28, 2015

Time Series Analysis and Forecasting23 references17 citations

TL;DR

This paper proposes Temporal Regularized Matrix Factorization (TRMF), a novel framework that integrates data-driven temporal dependency learning into matrix factorization for high-dimensional time series with missing values. By modeling temporal correlations through a learned transition matrix and leveraging low-rank factorization, TRMF achieves state-of-the-art performance in forecasting and imputation across real and synthetic datasets, outperforming classical and existing matrix factorization methods.

ABSTRACT

High-dimensional time series prediction is needed in applications as diverse as demand forecasting and climatology. Often, such applications require methods that are both highly scalable, and deal with noisy data in terms of corruptions or missing values. Classical time series methods usually fall short of handling both these issues. In this paper, we propose to adapt matrix matrix completion approaches that have previously been successfully applied to large scale noisy data, but which fail to adequately model high-dimensional time series due to temporal dependencies. We present a novel temporal regularized matrix factorization (TRMF) framework which supports data-driven temporal dependency learning and enables forecasting ability to our new matrix factorization approach. TRMF is highly general, and subsumes many existing matrix factorization approaches for time series data. We make interesting connections to graph regularized matrix factorization methods in the context of learning the dependencies. Experiments on both real and synthetic data show that TRMF outperforms several existing approaches for common time series tasks.

Motivation & Objective

• To address the limitations of classical time series models in handling high-dimensional, large-scale data with missing values.
• To develop a scalable matrix factorization framework that captures temporal dependencies in time series data.
• To enable effective forecasting and missing value imputation in high-dimensional time series without requiring complete data.
• To generalize existing matrix factorization and graph-regularized approaches by incorporating learnable temporal dynamics.

Proposed method

• TRMF formulates time series prediction as a low-rank matrix factorization problem, modeling the data matrix $ Y \in \mathbb{R}^{n \times T} $ as $ Y \approx FX $, where $ F $ captures time series-specific features and $ X $ captures time-varying latent states.
• A temporal regularizer is introduced to model dependencies among time points by learning a transition matrix $ M_\Theta $, which replaces the fixed or unstructured regularizers used in prior methods.
• The method optimizes a regularized objective function: $ \min_{F,X} \sum_{(i,t) \in \Omega} (Y_{it} - \mathbf{f}_i^\top \mathbf{x}_t)^2 + \lambda_f \mathcal{R}_f(F) + \lambda_x \mathcal{R}_x(X) $, where $ \Omega $ denotes observed entries.
• The temporal regularizer $ \mathcal{R}_x(X) $ is based on a learned transition matrix that encodes temporal dynamics, enabling data-driven dependency learning rather than relying on predefined graphs.
• TRMF supports both forecasting and imputation by leveraging the learned latent dynamics to predict future time points or reconstruct missing entries.
• The framework generalizes several existing methods, including AR models and graph-regularized MF, by subsuming them as special cases under different regularizer configurations.

Experimental results

Research questions

• RQ1 Can matrix factorization be effectively extended to model temporal dependencies in high-dimensional time series with missing values?
• RQ2 How can temporal dependencies be learned from data rather than imposed via fixed or graph-based structures?
• RQ3 Does a data-driven temporal regularizer in matrix factorization improve forecasting and imputation accuracy compared to classical time series models?
• RQ4 Can TRMF scale to large-scale, real-world time series datasets with missing data, such as Walmart sales or electricity consumption?

Key findings

• TRMF-AR outperforms all competing methods in 24-hour ahead forecasting on electricity and traffic datasets, with lower RMSE than SVD-AR(1), DLM, and TCF.
• On the Walmart-1 dataset, TRMF achieves a test RMSE of 1.239, significantly lower than competing methods such as DLM (1.453) and TCF (1.320).
• For missing value imputation, TRMF achieves the lowest RMSE across all tested datasets and missingness levels, with 50% missing data, TRMF achieved 0.184 RMSE on traffic data, outperforming DLM (0.224) and TCF (0.193).
• TRMF is scalable and stable on large-scale data, unlike R-DLM, which fails to run on $ n \geq 32 $, demonstrating TRMF's practical advantage in real-world applications.
• TRMF generalizes existing methods and subsumes AR models and graph-regularized MF as special cases, showing its flexibility and theoretical grounding.