[Paper Review] COMET: Convolutional Dimension Interaction for Collaborative Filtering

Zhuoyi Lin, Lei Feng+5 more|arXiv (Cornell University)|Jul 28, 2020

Recommender Systems and Techniques33 references4 citations

TL;DR

COMET proposes a novel collaborative filtering model that captures high-order interactions among historical user-item interactions and embedding dimensions using convolutional neural networks (CNNs) on stacked embedding maps. By modeling both internal (interaction-level) and dimensional (embedding-level) interactions simultaneously, COMET improves recommendation performance over state-of-the-art methods on implicit feedback datasets.

ABSTRACT

Representation learning-based recommendation models play a dominant role among recommendation techniques. However, most of the existing methods assume both historical interactions and embedding dimensions are independent of each other, and thus regrettably ignore the high-order interaction information among historical interactions and embedding dimensions. In this paper, we propose a novel representation learning-based model called COMET (COnvolutional diMEnsion inTeraction), which simultaneously models the high-order interaction patterns among historical interactions and embedding dimensions. To be specific, COMET stacks the embeddings of historical interactions horizontally at first, which results in two "embedding maps". In this way, internal interactions and dimensional interactions can be exploited by convolutional neural networks (CNN) with kernels of different sizes simultaneously. A fully-connected multi-layer perceptron (MLP) is then applied to obtain two interaction vectors. Lastly, the representations of users and items are enriched by the learnt interaction vectors, which can further be used to produce the final prediction. Extensive experiments and ablation studies on various public implicit feedback datasets clearly demonstrate the effectiveness and rationality of our proposed method.

Motivation & Objective

• To address the limitation of existing representation learning-based recommendation models that ignore high-order interactions between historical interactions and embedding dimensions.
• To model both internal interaction patterns (among historical interactions) and dimensional interaction patterns (among embedding dimensions) simultaneously.
• To enhance user and item representations by incorporating learned interaction signals without requiring auxiliary data like reviews or knowledge graphs.
• To demonstrate the effectiveness of convolutional networks in capturing complex, high-order interaction patterns in collaborative filtering.

Proposed method

• Stack the embeddings of a user’s or item’s historical interactions horizontally to form two 'embedding maps' that encode interaction-level and dimension-level patterns.
• Apply convolutional neural networks (CNNs) with multiple kernel sizes to extract high-order interaction signals from both embedding maps.
• Use a fully-connected multi-layer perceptron (MLP) to process the CNN outputs and generate two interaction vectors for users and items.
• Enrich the original user and item representations by concatenating them with the learned interaction vectors before computing the final prediction score.
• Train the model end-to-end using stochastic gradient descent with L2 regularization to prevent overfitting.
• Utilize outer-product operations implicitly through CNN kernels to model pairwise and higher-order dimensional interactions in a parameter-efficient manner.

Experimental results

Research questions

• RQ1 Can modeling high-order interactions between historical interactions and embedding dimensions improve recommendation performance in collaborative filtering?
• RQ2 How do internal (interaction-level) and dimensional (embedding-level) interactions contribute to representation learning in implicit feedback settings?
• RQ3 What is the relative contribution of original representations versus learned interaction vectors to final prediction accuracy?
• RQ4 How sensitive is the proposed method to hyperparameters such as embedding size, regularization, and number of CNN channels?

Key findings

• COMET outperforms state-of-the-art models on multiple public implicit feedback datasets in terms of HR@10 and NDCG@10.
• Ablation studies show that combining original representations with learned interaction vectors yields the best performance, confirming the value of interaction-aware representation learning.
• The model achieves improved performance with larger embedding sizes, indicating that richer representations enhance learning.
• Optimal regularization parameters lie in the range [5e-6, 1e-5], balancing overfitting and underfitting.
• Performance remains stable across different numbers of CNN channels, demonstrating the robustness and strong expressiveness of the CNN-based interaction modeling.
• Training loss converges steadily over epochs, indicating stable and effective optimization of the proposed architecture.