[Paper Review] Semiparametric Tensor Factor Analysis by Iteratively Projected SVD

Elynn Y. Chen, Dong Xia+2 more|arXiv (Cornell University)|Jul 5, 2020

Tensor decomposition and applications42 references15 citations

TL;DR

This paper proposes Semiparametric Tensor Factor Analysis (STEFA), a framework that enhances low-rank tensor decomposition by integrating auxiliary covariates into loading matrices via an Iteratively Projected SVD (IP-SVD) algorithm. IP-SVD improves estimation accuracy and convergence speed over Tucker decomposition, enabling effective estimation and prediction with new covariates on both synthetic and real tensor data.

ABSTRACT

This paper introduces a general framework of Semiparametric TEnsor FActor analysis (STEFA) that focuses on the methodology and theory of low-rank tensor decomposition with auxiliary covariates. STEFA models extend tensor factor models by incorporating instrumental covariates in the loading matrices. We propose an algorithm of Iteratively Projected SVD (IP-SVD) for the semiparametric estimations. It iteratively projects tensor data onto the linear space spanned by covariates and applies SVD on matricized tensors over each mode. We establish the convergence rates of the loading matrices and the core tensor factor. Compared with the Tucker decomposition, IP-SVD yields more accurate estimates with a faster convergence rate. Besides estimation, we show several prediction methods with newly observed covariates based on the STEFA model. On both real and synthetic tensor data, we demonstrate the efficacy of the STEFA model and the IP-SVD algorithm on both the estimation and prediction tasks.

Motivation & Objective

• To develop a flexible framework for low-rank tensor decomposition that incorporates auxiliary covariates to improve estimation accuracy.
• To address the limitation of traditional tensor factorization methods that do not leverage covariate information in loading matrices.
• To propose an efficient algorithm, IP-SVD, that iteratively projects tensor data onto covariate-induced subspaces for improved convergence and estimation.
• To establish theoretical convergence rates for loading matrices and core tensor factors under the STEFA model.
• To enable accurate prediction on new tensor data with unseen covariates using the fitted STEFA model.

Proposed method

• The STEFA model extends tensor factorization by embedding instrumental covariates into the loading matrices to capture covariate-dependent structure.
• The IP-SVD algorithm iteratively projects matricized tensors along each mode onto the linear space spanned by covariates before applying SVD.
• At each iteration, the method refines the low-rank approximation by leveraging the relationship between covariates and tensor modes.
• The algorithm alternates between projection steps and SVD-based updates to converge to a stable low-rank decomposition.
• Theoretical analysis establishes convergence rates for both loading matrices and the core tensor under the STEFA model.
• Prediction methods are derived based on the estimated model to handle new covariates not present during training.

Experimental results

Research questions

• RQ1 Can incorporating auxiliary covariates into loading matrices improve the accuracy of low-rank tensor decomposition?
• RQ2 How does the IP-SVD algorithm compare to Tucker decomposition in terms of estimation accuracy and convergence speed?
• RQ3 What are the theoretical convergence rates of the loading matrices and core tensor under the STEFA model?
• RQ4 Can the STEFA model support accurate prediction on new tensor data with previously unseen covariates?
• RQ5 How does the IP-SVD algorithm perform on both synthetic and real-world tensor datasets?

Key findings

• The IP-SVD algorithm achieves faster convergence and more accurate estimates compared to Tucker decomposition in low-rank tensor factorization.
• Theoretical convergence rates for loading matrices and core tensor factors are established under the STEFA model.
• The STEFA model enables effective prediction on new tensor data with novel covariates, extending its utility beyond estimation.
• Empirical results on synthetic and real tensor data confirm the superiority of IP-SVD in both estimation and prediction tasks.
• Incorporating covariates into loading matrices significantly enhances model interpretability and predictive performance.
• The iterative projection mechanism in IP-SVD effectively captures covariate-dependent structure across tensor modes.