[Paper Review] On Multivariate Singular Spectrum Analysis and its Variants

Anish K. Agarwal, Abdullah Alomar+1 more|arXiv (Cornell University)|Jun 24, 2020

Statistical and numerical algorithms4 citations

TL;DR

This paper introduces a novel variant of multivariate singular spectrum analysis (mSSA) under a spatio-temporal factor model, leveraging low-rank Hankel matrix structure and singular value thresholding to achieve prediction mean-squared error scaling as $1/\sqrt{\min(N,T)T}$ for both imputation and out-of-sample forecasting. The method improves upon classical SSA ($1/\sqrt{T}$) and matrix estimation ($1/\min(N,T)$) by exploiting temporal and cross-series structure.

ABSTRACT

We introduce and analyze a variant of multivariate singular spectrum analysis (mSSA), a popular time series method to impute and forecast a multivariate time series. Under a spatio-temporal factor model we introduce, given $N$ time series and $T$ observations per time series, we establish prediction mean-squared-error for both imputation and out-of-sample forecasting effectively scale as $1 / \sqrt{\min(N, T )T}$. This is an improvement over: (i) $1 /\sqrt{T}$ error scaling of SSA, the restriction of mSSA to a univariate time series; (ii) $1/\min(N, T)$ error scaling for matrix estimation methods which do not exploit temporal structure in the data. The spatio-temporal model we introduce includes any finite sum and products of: harmonics, polynomials, differentiable periodic functions, and Holder continuous functions. Our out-of-sample forecasting result could be of independent interest for online learning under a spatio-temporal factor model. Empirically, on benchmark datasets, our variant of mSSA performs competitively with state-of-the-art neural-network time series methods (e.g. DeepAR, LSTM) and significantly outperforms classical methods such as vector autoregression (VAR). Finally, we propose extensions of mSSA: (i) a variant to estimate time-varying variance of a time series; (ii) a tensor variant which has better sample complexity for certain regimes of $N$ and $T$.

Motivation & Objective

• To provide a rigorous finite-sample analysis of multivariate singular spectrum analysis (mSSA) for time series imputation and forecasting under missing data and noise.
• To establish theoretical error bounds that improve upon existing methods by leveraging temporal and cross-series correlation.
• To introduce a spatio-temporal factor model that captures harmonics, polynomials, periodic, and Hölder continuous functions.
• To develop algorithmic extensions for time-varying variance estimation and tensor-based mSSA with better sample complexity.

Proposed method

• Proposes a variant of mSSA that constructs a Hankel matrix from time series data and applies singular value thresholding (SVT) to recover low-rank structure.
• Uses a spatio-temporal factor model where each time series is a sum of products of latent components, enabling representation via low-rank tensors.
• Employs a Page-to-Hankel transformation to convert time series into a structured matrix form suitable for matrix estimation.
• Applies high-dimensional matrix estimation techniques, including singular value thresholding (HSTVT), to recover missing or noisy observations.
• Introduces a tensor variant of mSSA (tSSA) that models multivariate time series as a low-rank tensor, improving sample complexity in certain regimes.
• Develops a method for estimating time-varying variance by extending the core mSSA framework to model conditional heteroscedasticity.

Experimental results

Research questions

• RQ1 Can a theoretical finite-sample analysis of mSSA be established under a general spatio-temporal factor model with missing data and noise?
• RQ2 Does the proposed mSSA variant achieve better error scaling than classical SSA and matrix estimation methods?
• RQ3 How does the performance of the proposed mSSA variant compare to deep learning baselines like DeepAR and LSTM in imputation and forecasting tasks?
• RQ4 Can the mSSA framework be extended to estimate time-varying variance in time series?
• RQ5 Does the tensor variant of mSSA (tSSA) offer improved sample complexity compared to standard mSSA in high-dimensional settings?

Key findings

• The proposed mSSA variant achieves a prediction mean-squared error scaling of $1/\sqrt{\min(N,T)T}$, which improves upon the $1/\sqrt{T}$ scaling of univariate SSA.
• For the regime $N = o(T)$, the mSSA variant achieves error scaling $\tilde{\Theta}(1/\sqrt{NT})$, which is asymptotically better than the $\tilde{\Theta}(1/N)$ scaling of standard matrix estimation methods.
• In the regime $T = o(N)$, the mSSA variant achieves $\tilde{\Theta}(1/T)$ error, outperforming matrix estimation and tSSA.
• Empirical results show the method performs competitively with state-of-the-art neural network models like DeepAR and LSTM on benchmark datasets.
• The tensor variant of mSSA (tSSA) achieves better sample complexity than standard mSSA in certain regimes, particularly when $N$ is small relative to $T$.
• The method for time-varying variance estimation successfully captures non-constant conditional heteroscedasticity in time series, extending mSSA's applicability.