[Paper Review] Multilayer tensor factorization with applications to recommender systems

Xuan Bi, Annie Qu+1 more|arXiv (Cornell University)|Nov 5, 2017

Tensor decomposition and applications3 citations

TL;DR

This paper proposes REM (Recommendation Engine of Multilayers), a multilayer tensor factorization method for context-aware recommender systems that models user, item, and contextual factors through nested latent structures. By incorporating sub-group dependencies and a maximum block improvement algorithm, REM effectively addresses the cold-start problem and achieves superior performance on large-scale data, outperforming existing methods in simulations and real-world IRI marketing data with 116 million observations.

ABSTRACT

Recommender systems have been widely adopted by electronic commerce and entertainment industries for individualized prediction and recommendation, which benefit consumers and improve business intelligence. In this article, we propose an innovative method, namely the recommendation engine of multilayers (REM), for tensor recommender systems. The proposed method utilizes the structure of a tensor response to integrate information from multiple modes, and creates an additional layer of nested latent factors to accommodate between-subjects dependency. One major advantage is that the proposed method is able to address the "cold-start" issue in the absence of information from new customers, new products or new contexts. Specifically, it provides more effective recommendations through sub-group information. To achieve scalable computation, we develop a new algorithm for the proposed method, which incorporates a maximum block improvement strategy into the cyclic blockwise-coordinate-descent algorithm. In theory, we investigate both algorithmic properties for global and local convergence, along with the asymptotic consistency of estimated parameters. Finally, the proposed method is applied in simulations and IRI marketing data with 116 million observations of product sales. Numerical studies demonstrate that the proposed method outperforms existing competitors in the literature.

Motivation & Objective

• To address the cold-start problem in recommender systems where new users, items, or contexts lack sufficient historical data.
• To develop a scalable, context-aware tensor factorization method that integrates multiple modes of user, item, and contextual information.
• To model between-subjects dependencies through a nested latent factor structure to improve recommendation accuracy for new or sparse subjects.
• To ensure algorithmic scalability and convergence through a novel maximum block improvement strategy within a cyclic block-coordinate descent framework.
• To establish theoretical guarantees on global and local convergence, as well as asymptotic consistency of parameter estimates.

Proposed method

• The method models the tensor response as a multilayered structure with nested latent factors to capture hierarchical dependencies across users, items, and contexts.
• It employs a non-convex optimization framework based on cyclic blockwise coordinate descent enhanced by a maximum block improvement strategy for scalable computation.
• The model incorporates a tensor completion approach that leverages multi-modal data (e.g., user, item, time, location) to predict missing entries.
• A Hellinger metric entropy-based analysis is used to control model complexity and derive theoretical convergence rates.
• The algorithm is designed to handle higher-order tensors (beyond third-order), enabling integration of multiple contextual variables.
• Theoretical analysis includes proofs of global and local convergence, asymptotic consistency, and optimal rate of convergence under regularity conditions.

Experimental results

Research questions

• RQ1 Can a multilayer tensor factorization model effectively reduce the cold-start problem in recommender systems by leveraging sub-group information?
• RQ2 How does the proposed maximum block improvement strategy enhance convergence and scalability in non-convex tensor optimization?
• RQ3 What is the theoretical convergence behavior of the proposed algorithm, and does it achieve asymptotic consistency in parameter estimation?
• RQ4 How does the model perform on large-scale, real-world data with high-dimensional, sparse tensor structures?
• RQ5 Can the method outperform existing matrix and tensor-based recommender systems in terms of prediction accuracy and robustness?

Key findings

• The proposed REM method significantly outperforms existing competitors in both simulation studies and real-world IRI marketing data with 116 million observations.
• The method effectively mitigates the cold-start problem by utilizing sub-group information, enabling accurate recommendations even for new users, items, or contexts.
• Theoretical analysis confirms global and local convergence of the optimization algorithm, along with asymptotic consistency of estimated parameters.
• The optimal rate of convergence is derived as $ \varepsilon_{|\Omega|} \sim |\Omega|^{-1/2} $ under certain regularity conditions, indicating strong statistical efficiency.
• The maximum block improvement strategy enhances computational scalability, enabling efficient optimization on large-scale tensors.
• The model achieves superior performance on higher-order tensors by integrating multiple contextual variables beyond traditional two-mode systems.