[论文解读] Sampling-Decomposable Generative Adversarial Recommender

Binbin Jin, Defu Lian+5 more|arXiv (Cornell University)|Nov 2, 2020

Recommender Systems and Techniques参考文献 23被引用 14

一句话总结

该论文提出了一种新型框架——采样可分解生成对抗推荐模型（Sampling-Decomposable Generative Adversarial Recommender, SD-GAR），通过使用具有闭式优化和自归一化重要性采样以校正分布偏差的采样可分解生成器，显著提升了训练效率与推荐性能。SD-GAR在性能上比IRGAN高出12.4%，比当前最先进（SOTA）基线模型高出10%，且判别器训练速度最快可达20倍。

ABSTRACT

Recommendation techniques are important approaches for alleviating information overload. Being often trained on implicit user feedback, many recommenders suffer from the sparsity challenge due to the lack of explicitly negative samples. The GAN-style recommenders (i.e., IRGAN) addresses the challenge by learning a generator and a discriminator adversarially, such that the generator produces increasingly difficult samples for the discriminator to accelerate optimizing the discrimination objective. However, producing samples from the generator is very time-consuming, and our empirical study shows that the discriminator performs poor in top-k item recommendation. To this end, a theoretical analysis is made for the GAN-style algorithms, showing that the generator of limit capacity is diverged from the optimal generator. This may interpret the limitation of discriminator's performance. Based on these findings, we propose a Sampling-Decomposable Generative Adversarial Recommender (SD-GAR). In the framework, the divergence between some generator and the optimum is compensated by self-normalized importance sampling; the efficiency of sample generation is improved with a sampling-decomposable generator, such that each sample can be generated in O(1) with the Vose-Alias method. Interestingly, due to decomposability of sampling, the generator can be optimized with the closed-form solutions in an alternating manner, being different from policy gradient in the GAN-style algorithms. We extensively evaluate the proposed algorithm with five real-world recommendation datasets. The results show that SD-GAR outperforms IRGAN by 12.4% and the SOTA recommender by 10% on average. Moreover, discriminator training can be 20x faster on the dataset with more than 120K items.

研究动机与目标

• 解决尽管在top-k推荐中起关键作用，但GAN类推荐模型中判别器性能不佳的问题。
• 克服GAN类推荐模型因样本空间庞大而导致的样本生成计算成本过高的问题。
• 理论分析表明，容量有限的生成器会偏离最优分布，从而损害判别器的学习效果。
• 通过自归一化重要性采样补偿生成器的分布偏差，以提升推荐准确率。
• 利用采样可分解性实现生成器的高效闭式优化，避免GAN训练中策略梯度方法的低效问题。

提出的方法

• 提出一种采样可分解生成器，利用Vose-Alias方法实现O(1)时间复杂度的样本生成。
• 采用自归一化重要性采样，校正容量有限的生成器与最优生成器之间的分布偏差。
• 通过交替方式进行闭式解优化，避免在GAN类训练中使用策略梯度方法。
• 将生成器集成为动态负采样器，为预测分数更高的项目分配更高权重。
• 设计框架使得当生成器退化为均匀采样时，判别器可自然涵盖DNS和SA等现有模型，实现后向兼容。
• 通过解耦采样与生成器架构，实现可扩展训练，支持高效的每轮迭代更新。

实验结果

研究问题

• RQ1 为何在GAN类推荐模型中，尽管判别器是主要推理模型，其性能仍表现不佳？
• RQ2 容量有限的生成器的分布偏差如何影响判别器的推荐性能？
• RQ3 采样可分解性是否能够实现生成器的闭式优化，从而提升训练效率与稳定性？
• RQ4 自归一化重要性采样是否能在不依赖强化学习的前提下有效补偿生成器的分布误差？
• RQ5 所提出的框架在大规模数据集上能在多大程度上提升推荐准确率与训练速度？

主要发现

• 在五个真实世界数据集上，SD-GAR在NDCG指标上平均比IRGAN高出12.4%。
• SD-GAR在平均性能上比当前最先进（SOTA）推荐模型高出10%。
• 在包含超过12万个项目的数据集上，SD-GAR的判别器训练速度比IRGAN快达20倍。
• 由于实现O(1)采样与闭式优化，SD-GAR的生成器训练速度比IRGAN快5倍。
• SD-GAR中生成器与判别器的时间消耗随项目数量呈线性增长，而IRGAN的增长速率显著更陡峭。
• 当生成器退化为均匀采样时，判别器自然涵盖DNS和SA等模型，验证了该框架的后向兼容性。