[Paper Review] Learning Disentangled Representations for Recommendation

Jianxin Ma, Chang Zhou+3 more|arXiv (Cornell University)|Oct 31, 2019

Generative Adversarial Networks and Image SynthesisComputer Science100 citations

TL;DR

Introduces MacridVAE to learn macro and micro disentangled representations from user behavior, achieving improved recommendation performance and interpretable, controllable representations.

ABSTRACT

User behavior data in recommender systems are driven by the complex interactions of many latent factors behind the users' decision making processes. The factors are highly entangled, and may range from high-level ones that govern user intentions, to low-level ones that characterize a user's preference when executing an intention. Learning representations that uncover and disentangle these latent factors can bring enhanced robustness, interpretability, and controllability. However, learning such disentangled representations from user behavior is challenging, and remains largely neglected by the existing literature. In this paper, we present the MACRo-mIcro Disentangled Variational Auto-Encoder (MacridVAE) for learning disentangled representations from user behavior. Our approach achieves macro disentanglement by inferring the high-level concepts associated with user intentions (e.g., to buy a shirt or a cellphone), while capturing the preference of a user regarding the different concepts separately. A micro-disentanglement regularizer, stemming from an information-theoretic interpretation of VAEs, then forces each dimension of the representations to independently reflect an isolated low-level factor (e.g., the size or the color of a shirt). Empirical results show that our approach can achieve substantial improvement over the state-of-the-art baselines. We further demonstrate that the learned representations are interpretable and controllable, which can potentially lead to a new paradigm for recommendation where users are given fine-grained control over targeted aspects of the recommendation lists.

Motivation & Objective

• Motivate the need for disentangled representations in recommender systems to improve robustness and interpretability.
• Propose a MACRo-mIcro Disentangled Variational Auto-Encoder (MacridVAE) that separates high-level intentions from low-level item attributes.
• Develop macro disentanglement through concept prototypes and category assignments, and micro disentanglement via a beta-regularized KL objective.
• Demonstrate that learned representations are interpretable and offer user controllability over recommendations.

Proposed method

• Propose a generative model where user interactions x_u are generated from latent z_u and item concept indicators C.
• Implement macro disentanglement by splitting z_u into K concept-specific parts z_u^(k) and associating items with one-hot concept vectors c_i via prototypes m_k.
• Use a variational encoder q_theta(z_u|x_u,C) and a cosine-based prototype assignment to prevent mode collapse.
• Apply a beta-VAE style objective that strengthens the KL term to promote micro-disentanglement and independence across dimensions.
• Use sampled softmax for scalable decoding over large item sets, with g_theta^(i)(z_u^(k)) guiding the likelihood of item i within concept k.
• Adopt prior p_theta(z_u) as Gaussian and a cosine-based decoder to align micro and macro factors.
• Provide a beam-search based method for user-controllable recommendations to navigate gradual changes in latent dimensions.

Experimental results

Research questions

• RQ1 Can macro-level disentanglement separate user intentions by concept while preserving cross-concept preferences?
• RQ2 Can micro-level disentanglement yield independent latent factors per dimension and improve recommendation robustness?
• RQ3 Does incorporating macro and micro disentanglement improve predictive performance on real-world datasets?
• RQ4 Are the learned representations interpretable and controllable by users?
• RQ5 Does cosine-based prototype assignment mitigate mode collapse compared to inner-product schemes?

Key findings

• MacridVAE achieves substantial improvements over state-of-the-art baselines on five real-world datasets, especially for sparse, small datasets.
• Macro disentanglement enables diverse user interest modeling and helps alleviate data sparsity by sharing information within concepts.
• Micro disentanglement regularization yields more independent latent dimensions, enhancing robustness and interpretability.
• Using cosine similarity for prototype assignment prevents mode collapse and yields more meaningful concept clustering than inner product.
• The learned representations are interpretable, with dimensions aligning to human-understandable concepts, and enable potential user-controlled recommendation.