[Paper Review] Embedded Collaborative Filtering for "Cold Start" Prediction

Yubo Zhou, Ali Nadaf|arXiv (Cornell University)|Apr 9, 2017

Recommender Systems and Techniques9 references3 citations

TL;DR

This paper proposes Embedded Collaborative Filtering (ECF), a hybrid method that leverages Word2Vec (specifically Skip-gram and CBOW) for dimensionality reduction on implicit user-item interaction data to improve recommendation performance in 'Cold Start' scenarios. By learning dense distributed representations of items from short-term user sessions, ECF outperforms traditional collaborative filtering and state-of-the-art methods in precision for top-1 and top-5 recommendations when user interaction history is minimal.

ABSTRACT

Using only implicit data, many recommender systems fail in general to provide a precise set of recommendations to users with limited interaction history. This issue is regarded as the "Cold Start" problem and is typically resolved by switching to content-based approaches where extra costly information is required. In this paper, we use a dimensionality reduction algorithm, Word2Vec (W2V), originally applied in Natural Language Processing problems under the framework of Collaborative Filtering (CF) to tackle the "Cold Start" problem using only implicit data. This combined method is named Embedded Collaborative Filtering (ECF). An experiment is conducted to determine the performance of ECF on two different implicit data sets. We show that the ECF approach outperforms other popular and state-of-the-art approaches in "Cold Start" scenarios.

Motivation & Objective

• To address the 'Cold Start' problem in recommender systems where users have minimal interaction history.
• To improve recommendation accuracy using only implicit feedback data without requiring auxiliary information such as user or item metadata.
• To develop a method that leverages distributed representation learning (Word2Vec) within a collaborative filtering framework to model user preferences from sparse interactions.
• To evaluate the proposed method against state-of-the-art approaches in both 'Cold Start' and non-'Cold Start' scenarios.

Proposed method

• The method applies Word2Vec (Skip-gram and CBOW) to learn low-dimensional dense embeddings of items from user interaction sequences, treating each user session as a 'sentence' of items.
• Item embeddings are learned using negative sampling and stochastic gradient descent to maximize the likelihood of co-occurring items in user sessions.
• The model uses these learned item embeddings to compute item-item similarities via cosine similarity, enabling collaborative filtering recommendations.
• A hybrid model is constructed by combining short-term (window size 5) and long-term (window size 20) models, with weighted averaging to improve robustness.
• Recommendations are generated by identifying the k-nearest neighbor items based on similarity scores derived from embedded item vectors.
• The approach is evaluated using precision@1 and precision@5 metrics across two public datasets under varying levels of data sparsity.

Experimental results

Research questions

• RQ1 Can Word2Vec-based item embedding improve collaborative filtering performance in 'Cold Start' scenarios with only implicit feedback?
• RQ2 How does the performance of ECF compare to traditional Item-Item KNN and other state-of-the-art methods under high data sparsity?
• RQ3 Does combining short-term and long-term user session models lead to better recommendation accuracy than single-model approaches?
• RQ4 Which Word2Vec variant—Skip-gram or CBOW—yields better performance in the context of implicit feedback-based recommendation?
• RQ5 How does the rate of hidden (unseen) items in user sessions affect the precision of ECF and baseline methods?

Key findings

• ECF outperforms traditional Item-Item KNN and other state-of-the-art methods in precision@1 and precision@5 for 'Cold Start' scenarios, especially when 90% or 95% of items are hidden.
• On the MovieLens 100k dataset, ECF with Skip-gram achieved a precision@1 of 0.436 at 90% hidden rate, surpassing CF (0.316) and CDAE (0.379).
• The hybrid model combining short-term and long-term user session embeddings achieved the highest precision, indicating that multi-scale modeling improves robustness.
• Skip-gram consistently outperformed CBOW across both datasets and all hidden rates, suggesting better representation learning for sequential item patterns.
• Precision increased with higher hidden item rates, indicating that ECF is particularly effective when the system must predict for users with very limited interaction history.
• In non-'Cold Start' scenarios, traditional CF and CDAE methods performed better than ECF, confirming that ECF's advantage is most pronounced in low-data regimes.