[Paper Review] Denoising User-aware Memory Network for Recommendation

Zhi Bian, Shaojun Zhou+7 more|arXiv (Cornell University)|Jul 12, 2021

Recommender Systems and Techniques45 references4 citations

TL;DR

This paper proposes DUMN, a denoising user-aware memory network that improves CTR prediction by purifying noisy implicit feedback using explicit feedback via orthogonal mapping, modeling long-term user preferences with a refined memory network, and fusing short- and long-term interests through a gating mechanism. DUMN achieves state-of-the-art performance on two real-world e-commerce datasets, demonstrating superior accuracy and robustness in capturing unbiased user preferences.

ABSTRACT

For better user satisfaction and business effectiveness, more and more attention has been paid to the sequence-based recommendation system, which is used to infer the evolution of users' dynamic preferences, and recent studies have noticed that the evolution of users' preferences can be better understood from the implicit and explicit feedback sequences. However, most of the existing recommendation techniques do not consider the noise contained in implicit feedback, which will lead to the biased representation of user interest and a suboptimal recommendation performance. Meanwhile, the existing methods utilize item sequence for capturing the evolution of user interest. The performance of these methods is limited by the length of the sequence, and can not effectively model the long-term interest in a long period of time. Based on this observation, we propose a novel CTR model named denoising user-aware memory network (DUMN). Specifically, the framework: (i) proposes a feature purification module based on orthogonal mapping, which use the representation of explicit feedback to purify the representation of implicit feedback, and effectively denoise the implicit feedback; (ii) designs a user memory network to model the long-term interests in a fine-grained way by improving the memory network, which is ignored by the existing methods; and (iii) develops a preference-aware interactive representation component to fuse the long-term and short-term interests of users based on gating to understand the evolution of unbiased preferences of users. Extensive experiments on two real e-commerce user behavior datasets show that DUMN has a significant improvement over the state-of-the-art baselines. The code of DUMN model has been uploaded as an additional material.

Motivation & Objective

• To address the issue of noise in implicit feedback, which biases user preference representation and degrades recommendation performance.
• To model long-term user interests more effectively, overcoming the limitations of sequence-length-dependent RNN-based methods.
• To improve the accuracy of CTR prediction by fusing short-term and long-term interests in a preference-aware manner.
• To develop a unified framework that leverages both implicit (click/unclick) and explicit (like/dislike) feedback for unbiased preference modeling.
• To demonstrate the effectiveness of feature purification and fine-grained long-term interest representation in real-world recommendation scenarios.

Proposed method

• Introduces a feature purification (FP) module that uses orthogonal mapping to align and refine implicit feedback representations based on explicit feedback, reducing noise.
• Proposes a user memory network (UMN) that enhances memory capacity and stability to capture long-term user interests over extended periods.
• Designs a preference-aware interactive representation (PAIR) module that dynamically fuses short-term and long-term interests using a gating mechanism.
• Constructs four feedback sequences—click, unclick, like, and dislike—for comprehensive user preference modeling.
• Employs a denoising strategy that leverages the higher reliability of explicit feedback to correct noisy implicit feedback representations.
• Trains the end-to-end model using a multi-task learning objective to jointly optimize CTR prediction and dislike prediction.

Experimental results

Research questions

• RQ1 Can explicit feedback be effectively leveraged to denoise implicit feedback representations and improve user preference modeling?
• RQ2 Can a refined memory network architecture better capture and retain long-term user interests compared to standard RNNs?
• RQ3 Does the fusion of short-term and long-term interests through a gating mechanism lead to more accurate and stable CTR predictions?
• RQ4 How does the proposed DUMN model perform compared to state-of-the-art baselines in real-world e-commerce recommendation tasks?
• RQ5 To what extent does the model improve in capturing users' disinterest (e.g., dislike prediction), reducing user frustration?

Key findings

• DUMN achieves significant improvements over state-of-the-art baselines on two real e-commerce datasets, demonstrating superior CTR prediction performance.
• The feature purification module effectively reduces noise in implicit feedback, as shown by the increased separation between denoised click/unclick representations and explicit like/dislike representations in visualization.
• The user memory network successfully captures stable long-term user interests, with representations of the same user clustering together in the embedding space over time.
• The short-term interest representations are more dispersed, while long-term representations converge to a stable, compact region, confirming the model’s ability to distinguish between transient and enduring preferences.
• DUMN outperforms the DFN baseline in dislike prediction, indicating enhanced sensitivity to user disinterest and improved model robustness.
• Visualization results confirm that the FP and UMN modules effectively enhance the model’s ability to learn unbiased, fine-grained user preferences.