[Paper Review] Joint Deep Modeling of Users and Items Using Reviews for Recommendation

Lei Zheng, Vahid Noroozi+1 more|arXiv (Cornell University)|Jan 17, 2017

Recommender Systems and TechniquesComputer Science31 references86 citations

TL;DR

DeepCoNN jointly models users and items from review text using two CNNs with a shared interaction layer, achieving superior rating prediction over baselines across multiple datasets.

ABSTRACT

A large amount of information exists in reviews written by users. This source of information has been ignored by most of the current recommender systems while it can potentially alleviate the sparsity problem and improve the quality of recommendations. In this paper, we present a deep model to learn item properties and user behaviors jointly from review text. The proposed model, named Deep Cooperative Neural Networks (DeepCoNN), consists of two parallel neural networks coupled in the last layers. One of the networks focuses on learning user behaviors exploiting reviews written by the user, and the other one learns item properties from the reviews written for the item. A shared layer is introduced on the top to couple these two networks together. The shared layer enables latent factors learned for users and items to interact with each other in a manner similar to factorization machine techniques. Experimental results demonstrate that DeepCoNN significantly outperforms all baseline recommender systems on a variety of datasets.

Motivation & Objective

• Leverage abundant review text to alleviate sparsity in recommender systems.
• Learn user behaviors and item properties jointly from reviews.
• Couple user and item latent factors in a common prediction framework.
• Evaluate whether joint modeling with text outperforms rating-only baselines across diverse datasets.

Proposed method

• Two parallel CNN-based networks (Net_u for users, Net_i for items) process merged reviews into latent features.
• Word embeddings are used to preserve word order and capture semantic information.
• A shared layer using a Factorization Machine aggregates x_u and y_i to predict ratings.
• Training minimizes a joint objective that models second-order interactions between user and item features.
• Pre-trained word embeddings (from Google News) initialize the input layer and are fine-tuned during training.
• Dropout and RMSprop are used to prevent overfitting and optimize learning.

Experimental results

Research questions

• RQ1 Can jointly modeling user and item reviews improve rating prediction beyond baselines that use only ratings or reviews separately?
• RQ2 Does preserving word order via embeddings and using a shared interaction layer yield better performance than topic-modeling based approaches?
• RQ3 Is the model scalable for online learning and capable of handling sparse data (cold-start scenarios)?

Key findings

• DeepCoNN outperforms MF, PMF, LDA, CTR, HFT, and CDL baselines on Yelp, Amazon, and Beer in terms of lower MSE.
• Average MSE across datasets: MF 1.292, PMF 1.256, LDA 1.184, CTR 1.112, HFT-10 1.088, HFT-50 1.09, CDL 1.081, DeepCoNN 0.994 (Best).
• DeepCoNN yields an average improvement of 8.3% over the best baseline.
• DeepCoNN shows stronger gains for users/items with fewer ratings, helping alleviate sparsity and cold-start problems.
• Variants analysis shows modeling both users and items with word embeddings and the shared FM layer is advantageous over single-network or TF-IDF/Random representations.
• DeepCoNN remains effective across diverse domains (Yelp, Amazon categories, Beer).