[Paper Review] Adversarial and Contrastive Variational Autoencoder for Sequential Recommendation

Zhe Xie, Chengxuan Liu+4 more|arXiv (Cornell University)|Mar 19, 2021

Recommender Systems and Techniques31 references4 citations

TL;DR

This paper proposes ACVAE, a novel sequential recommendation model that enhances variational autoencoders by integrating adversarial training via Adversarial Variational Bayes (AVB) and contrastive learning. By reducing inter-dimensional correlation in latent variables and improving personalized representation learning, ACVAE achieves state-of-the-art performance on four real-world datasets, outperforming existing SOTA methods in recall and NDCG metrics.

ABSTRACT

Sequential recommendation as an emerging topic has attracted increasing attention due to its important practical significance. Models based on deep learning and attention mechanism have achieved good performance in sequential recommendation. Recently, the generative models based on Variational Autoencoder (VAE) have shown the unique advantage in collaborative filtering. In particular, the sequential VAE model as a recurrent version of VAE can effectively capture temporal dependencies among items in user sequence and perform sequential recommendation. However, VAE-based models suffer from a common limitation that the representational ability of the obtained approximate posterior distribution is limited, resulting in lower quality of generated samples. This is especially true for generating sequences. To solve the above problem, in this work, we propose a novel method called Adversarial and Contrastive Variational Autoencoder (ACVAE) for sequential recommendation. Specifically, we first introduce the adversarial training for sequence generation under the Adversarial Variational Bayes (AVB) framework, which enables our model to generate high-quality latent variables. Then, we employ the contrastive loss. The latent variables will be able to learn more personalized and salient characteristics by minimizing the contrastive loss. Besides, when encoding the sequence, we apply a recurrent and convolutional structure to capture global and local relationships in the sequence. Finally, we conduct extensive experiments on four real-world datasets. The experimental results show that our proposed ACVAE model outperforms other state-of-the-art methods.

Motivation & Objective

• To address the limited representational capacity of approximate posterior distributions in VAE-based sequential recommendation models.
• To improve the quality and diversity of generated latent variables for better sequence generation and recommendation.
• To enhance personalized user representation by minimizing correlation between latent dimensions and maximizing inter-user discriminability.
• To capture both long-term and short-term sequential patterns using a hybrid RNN-CNN encoder architecture.
• To validate the effectiveness of adversarial and contrastive learning components in improving recommendation performance.

Proposed method

• Introduces adversarial training within the Adversarial Variational Bayes (AVB) framework to regularize latent variables and reduce inter-dimensional correlation.
• Employs a contrastive loss to encourage distinct latent representations for different users, enhancing personalization and generalization.
• Uses a recurrent encoder (LSTM) to model long-term temporal dependencies in user interaction sequences.
• Applies a 1D convolutional layer after the RNN to capture local, short-term patterns in the sequence.
• Optimizes the joint objective combining VAE reconstruction, KL divergence, adversarial loss, and contrastive loss.
• Leverages a discriminator network to distinguish real from generated latent variables, improving the quality of the posterior approximation.

Experimental results

Research questions

• RQ1 Can adversarial training via AVB improve the quality and diversity of latent variables in sequential VAEs?
• RQ2 Does contrastive learning enhance the model's ability to distinguish personalized user representations in sequential recommendation?
• RQ3 How do the combination of adversarial and contrastive learning affect recommendation performance compared to standard VAEs?
• RQ4 What is the impact of the CNN layer in capturing local sequential patterns after the RNN encoder?
• RQ5 How do hyperparameters α (adversarial weight) and β (contrastive loss weight) affect model performance?

Key findings

• ACVAE outperforms state-of-the-art models on four real-world datasets in both Recall@10 and NDCG@10 metrics.
• The ablation study shows that removing AVB leads to a significant drop in performance, confirming its role in improving latent variable quality.
• The correlation coefficient between latent dimensions is significantly reduced when AVB is applied, indicating lower inter-dimensional dependency.
• Contrastive learning improves model generalization, with optimal performance achieved at β = 0.5, especially when KL divergence is weak.
• The inclusion of the CNN layer after the RNN improves performance by capturing local sequential patterns more effectively.
• Hyperparameter analysis shows that α = 0.05 and β = 0.5 yield the best trade-off between regularization and representation learning.