[論文レビュー] Variational Autoencoders for Collaborative Filtering

Dawen Liang, Rahul G. Krishnan+2 more|arXiv (Cornell University)|Feb 16, 2018

Recommender Systems and Techniques参考文献 46被引用数 66

ひとこと要約

この論文は変分オートエンコーダを協調フィルタリングへ拡張し、implicit feedback に対して multinomial 尤度と beta-正規化された目的関数を用いて、実データセットで最先端の結果を達成している。

ABSTRACT

We extend variational autoencoders (VAEs) to collaborative filtering for implicit feedback. This non-linear probabilistic model enables us to go beyond the limited modeling capacity of linear factor models which still largely dominate collaborative filtering research.We introduce a generative model with multinomial likelihood and use Bayesian inference for parameter estimation. Despite widespread use in language modeling and economics, the multinomial likelihood receives less attention in the recommender systems literature. We introduce a different regularization parameter for the learning objective, which proves to be crucial for achieving competitive performance. Remarkably, there is an efficient way to tune the parameter using annealing. The resulting model and learning algorithm has information-theoretic connections to maximum entropy discrimination and the information bottleneck principle. Empirically, we show that the proposed approach significantly outperforms several state-of-the-art baselines, including two recently-proposed neural network approaches, on several real-world datasets. We also provide extended experiments comparing the multinomial likelihood with other commonly used likelihood functions in the latent factor collaborative filtering literature and show favorable results. Finally, we identify the pros and cons of employing a principled Bayesian inference approach and characterize settings where it provides the most significant improvements.

研究の動機と目的

• Extend variational autoencoders to collaborative filtering for implicit feedback.
• Adopt a multinomial likelihood to better model click data and ranking loss.
• Introduce a beta-regularized ELBO with annealing to improve performance.
• Compare multinomial VAE to Gaussian and logistic baselines and analyze Bayesian inference benefits.

提案手法

• Generative model: latent z_u ~ N(0,I) mapped via a non-linear function f_theta to a probability over I items, then x_u ~ Mult(N_u, pi(z_u)).
• Inference: amortized variational posterior q_phi(z_u|x_u) ~ N(mu_phi(x_u), diag(sigma_phi^2(x_u))).
• Optimization: maximize the evidence lower bound ELBO with a reparameterization trick for gradients; introduce beta to regularize KL term.
• Beta-regularized objective: L_beta = E_q[log p_theta(x_u|z_u)] - beta * KL(q_phi(z_u|x_u)||p(z_u)); anneal beta from 0 to 1 to improve training.
• Prediction: rank items by unnormalized multinomial probabilities using z_u = mu_phi(x_u) (or z from the encoder in Mult-DAE).
• Notes: address computational considerations for large item sets and relate approach to autoencoder taxonomy.

実験結果

リサーチクエスチョン

• RQ1 Does a multinomial likelihood improve ranking-oriented evaluation over Gaussian or logistic likelihoods in implicit-feedback data?
• RQ2 Can a beta-regularized VAE with KL annealing yield better ranking metrics than standard VAE training in collaborative filtering?
• RQ3 How does Mult-VAE-pr compare to denoising autoencoders and other neural CF methods on real-world datasets?
• RQ4 What are the trade-offs of explicit uncertainty modeling (Mult-VAE-pr) vs deterministic encoders (Mult-DAE) in recommendations?

主な発見

Dataset	ML-20M	Netflix	MSD
# of users	136,677	463,435	571,355
# of items	20,108	17,769	41,140
# of interactions	10.0M	56.9M	33.6M
% of interactions	0.36%	0.69%	0.14%
# of held-out users	10,000	40,000	50,000

• Mult-VAE-pr achieves state-of-the-art results on several real-world datasets compared to baselines including recent neural-network approaches.
• The multinomial likelihood consistently outperforms Gaussian and logistic likelihoods for implicit feedback in this setting.
• Both Mult-VAE-pr and Mult-DAE provide competitive performance, with explicit uncertainty modeling offering advantages in certain regimes.
• Beta annealing is crucial for avoiding underutilization of the latent space and yields significant performance gains; best results typically obtained by annealing then selecting the peak validation metric.
• The approach connects to information-theoretic ideas (information bottleneck, maximum-entropy discrimination) and remains robust under data sparsity.