[Paper Review] Scalable Recommendation with Poisson Factorization

Prem Gopalan, Jake M. Hofman+1 more|arXiv (Cornell University)|Nov 7, 2013

Recommender Systems and TechniquesComputer Science48 references114 citations

TL;DR

This paper proposes Poisson Factorization (PF) and its hierarchical variant HPF for scalable, high-accuracy recommendation systems. By modeling user-item interactions as Poisson-distributed counts, PF naturally captures sparse, finite user consumption behavior—outperforming nonnegative matrix factorization, LDA, and probabilistic matrix factorization across explicit and implicit feedback data, with HPF achieving up to 8 percentage points higher normalized precision at 20 recommendations.

ABSTRACT

We develop a Bayesian Poisson matrix factorization model for forming recommendations from sparse user behavior data. These data are large user/item matrices where each user has provided feedback on only a small subset of items, either explicitly (e.g., through star ratings) or implicitly (e.g., through views or purchases). In contrast to traditional matrix factorization approaches, Poisson factorization implicitly models each user's limited attention to consume items. Moreover, because of the mathematical form of the Poisson likelihood, the model needs only to explicitly consider the observed entries in the matrix, leading to both scalable computation and good predictive performance. We develop a variational inference algorithm for approximate posterior inference that scales up to massive data sets. This is an efficient algorithm that iterates over the observed entries and adjusts an approximate posterior over the user/item representations. We apply our method to large real-world user data containing users rating movies, users listening to songs, and users reading scientific papers. In all these settings, Bayesian Poisson factorization outperforms state-of-the-art matrix factorization methods.

Motivation & Objective

• To address the limitations of traditional matrix factorization in modeling sparse, finite user consumption behavior in real-world recommendation systems.
• To develop a scalable probabilistic model that naturally handles both explicit ratings and implicit feedback (e.g., clicks, views) without ad hoc modifications.
• To improve recommendation accuracy by accounting for user-specific budgets and item popularity heterogeneity through a Poisson likelihood.
• To enable efficient inference on massive datasets using variational inference, ensuring scalability to billion-scale user-item matrices.
• To demonstrate consistent superiority of PF and HPF across diverse real-world datasets, including Netflix, Last.FM, Mendeley, and NYT.

Proposed method

• Models user-item interactions as Poisson-distributed counts, where the rate parameter is the dot product of user preference and item attribute latent vectors.
• Uses a hierarchical prior over user and item latent factors to allow for flexible, data-driven modeling of user heterogeneity and item popularity.
• Employs a variational inference algorithm with stochastic optimization to scale to massive datasets, iterating only over observed (non-zero) entries.
• Applies a two-stage generative process: users first choose a budget (number of items to consume), then allocate it across preferred items based on latent attributes.
• Incorporates non-negativity and sparsity constraints on latent factors via appropriate priors, aligning with real-world user behavior patterns.
• Uses posterior predictive checks to validate model fit and demonstrate that PF avoids the overestimation of user budgets seen in classical matrix factorization.

Experimental results

Research questions

• RQ1 Can a probabilistic matrix factorization model based on the Poisson distribution better capture real-world user behavior than existing methods?
• RQ2 Does Poisson factorization outperform nonnegative matrix factorization, LDA, and standard matrix factorization on both explicit and implicit feedback data?
• RQ3 How does the model handle user heterogeneity in consumption behavior and item popularity across diverse data domains?
• RQ4 Can the hierarchical variant HPF provide improved performance and robustness across users of varying activity levels?
• RQ5 Does the model’s generative process, which accounts for finite user budgets, lead to better predictive performance and reduced need for data-specific tuning?

Key findings

• HPF and BPF outperformed all baseline methods on all four datasets (Netflix, Last.FM, Mendeley, NYT), achieving up to 8 percentage points higher normalized precision at 20 recommendations.
• Poisson factorization significantly outperformed classical matrix factorization with biases (MF), especially on implicit feedback data, without requiring ad hoc downweighting of zeros.
• The model maintained superior performance across all user activity levels, including the least active 10%, demonstrating robustness to user sparsity.
• Posterior predictive checks confirmed that classical matrix factorization systematically overestimates user budgets, leading to overweighting of unobserved (zero) entries—a flaw avoided by PF.
• Exploratory analysis revealed that learned components in HPF capture meaningful, interpretable topics that cut across and differentiate between conventional categories, such as separating self-help and personal finance in news articles.
• The algorithm scaled efficiently to massive datasets, including the Netflix dataset with 8.16 billion cells and 250 million ratings, using only observed entries in inference.