[Paper Review] Low-Rank Autoregressive Tensor Completion for Multivariate Time Series Forecasting

Xinyu Chen, Lijun Sun|arXiv (Cornell University)|Jun 18, 2020

Tensor decomposition and applications30 references20 citations

TL;DR

This paper proposes Low-Rank Autoregressive Tensor Completion (LATC), a framework that transforms multivariate time series into a third-order tensor (sensor × time of day × day) to jointly model global low-rank patterns and local autoregressive trends. LATC achieves state-of-the-art performance in both missing data imputation and rolling prediction across three real-world datasets by integrating tensor nuclear norm minimization and a novel autoregressive norm.

ABSTRACT

Time series prediction has been a long-standing research topic and an essential application in many domains. Modern time series collected from sensor networks (e.g., energy consumption and traffic flow) are often large-scale and incomplete with considerable corruption and missing values, making it difficult to perform accurate predictions. In this paper, we propose a low-rank autoregressive tensor completion (LATC) framework to model multivariate time series data. The key of LATC is to transform the original multivariate time series matrix (e.g., sensor$ imes$time point) to a third-order tensor structure (e.g., sensor$ imes$time of day$ imes$day) by introducing an additional temporal dimension, which allows us to model the inherent rhythms and seasonality of time series as global patterns. With the tensor structure, we can transform the time series prediction and missing data imputation problems into a universal low-rank tensor completion problem. Besides minimizing tensor rank, we also integrate a novel autoregressive norm on the original matrix representation into the objective function. The two components serve different roles. The low-rank structure allows us to effectively capture the global consistency and trends across all the three dimensions (i.e., similarity among sensors, similarity of different days, and current time v.s. the same time of historical days). The autoregressive norm can better model the local temporal trends. Our numerical experiments on three real-world data sets demonstrate the superiority of the integration of global and local trends in LATC in both missing data imputation and rolling prediction tasks.

Motivation & Objective

• Address the challenge of accurate forecasting in large-scale, incomplete multivariate time series with missing values and corruption.
• Overcome limitations of traditional models that struggle with both global temporal patterns and local dynamics in high-dimensional, incomplete data.
• Develop a unified framework that treats imputation and prediction as a single tensor completion problem.
• Integrate global low-rank structure across sensor, time-of-day, and day dimensions with local autoregressive dynamics to improve predictive accuracy.
• Enable all observed data to contribute to final predictions by leveraging the tensor structure for better modeling of complex spatiotemporal dependencies.

Proposed method

• Transform the original multivariate time series matrix (sensor × time point) into a third-order tensor (sensor × time of day × day) to capture daily and weekly seasonality.
• Apply tensor nuclear norm minimization and truncated nuclear norm minimization to enforce low-rank structure across all three dimensions, modeling global consistency and long-term trends.
• Introduce a novel autoregressive norm on the original matrix representation to capture short-term, local temporal dynamics and fine-grained temporal correlations.
• Formulate the objective function as a combination of low-rank tensor completion and autoregressive regularization, enabling joint optimization.
• Use alternating direction method of multipliers (ADMM) or similar optimization techniques to solve the non-convex, non-smooth optimization problem.
• Train the model end-to-end on observed data to predict missing values and future time points, with the prediction window defined by the user.

Experimental results

Research questions

• RQ1 Can transforming multivariate time series into a tensor structure improve modeling of both global and local temporal patterns in the presence of missing data?
• RQ2 How does the integration of low-rank tensor completion and an autoregressive norm compare to using either component alone in time series forecasting?
• RQ3 To what extent can LATC outperform state-of-the-art matrix- and tensor-based models in both imputation and rolling prediction tasks?
• RQ4 Does the tensor representation allow for better utilization of all observed data, including historical patterns across days and times of day, compared to matrix-based approaches?
• RQ5 Can LATC effectively handle large-scale, high-dimensional time series with significant missingness while maintaining computational feasibility?

Key findings

• LATC achieves the best imputation accuracy across all three real-world datasets, outperforming both low-rank tensor models (e.g., HaLRTC) and matrix-based models (e.g., TRMF, BTMF).
• The integration of the autoregressive norm significantly improves performance over pure low-rank models, as shown by LATC-TNN outperforming HaLRTC-TNN in all imputation and prediction tasks.
• In rolling prediction, LATC-TNN achieves a mean MAPE of 6.39 and RMSE of 4.97 on the original data (P), outperforming TRMF (11.30/7.19) and BTMF (8.83/5.95) by a substantial margin.
• On the ETT dataset with 40% missing values under non-MAR (NM) conditions, LATC-TNN achieves a MAPE of 7.88 and RMSE of 5.65, significantly outperforming the next-best model (12.67/4.87).
• The results demonstrate that modeling both global (low-rank) and local (autoregressive) patterns is essential for robust performance in incomplete time series forecasting.
• LATC-TNN consistently outperforms its NN-based counterpart (LATC-NN), confirming the effectiveness of the low-rank tensor structure in capturing long-term dependencies.