[Paper Review] Adversarial Recommendation: Attack of the Learned Fake Users

Konstantina Christakopoulou, Arindam Banerjee|arXiv (Cornell University)|Sep 21, 2018

Adversarial Robustness in Machine Learning28 references3 citations

TL;DR

This paper introduces a machine learning-based framework for generating realistic fake user profiles that can adversarially manipulate recommendation systems while remaining undetectable. By formulating the attack as a two-player game between a recommender system and a GAN-based adversary, the method optimizes both profile realism and attack intent, demonstrating that even subtle, learned fake users can significantly degrade recommendation quality for targeted users.

ABSTRACT

Can machine learning models for recommendation be easily fooled? While the question has been answered for hand-engineered fake user profiles, it has not been explored for machine learned adversarial attacks. This paper attempts to close this gap. We propose a framework for generating fake user profiles which, when incorporated in the training of a recommendation system, can achieve an adversarial intent, while remaining indistinguishable from real user profiles. We formulate this procedure as a repeated general-sum game between two players: an oblivious recommendation system $R$ and an adversarial fake user generator $A$ with two goals: (G1) the rating distribution of the fake users needs to be close to the real users, and (G2) some objective $f_A$ encoding the attack intent, such as targeting the top-K recommendation quality of $R$ for a subset of users, needs to be optimized. We propose a learning framework to achieve both goals, and offer extensive experiments considering multiple types of attacks highlighting the vulnerability of recommendation systems.

Motivation & Objective

• To investigate whether machine learning models for recommendation can be manipulated by adversarial fake user profiles that are indistinguishable from real users.
• To develop a learning framework that generates fake user profiles with realistic rating distributions while optimizing for specific attack objectives, such as degrading top-K recommendations for targeted users.
• To bridge the gap between traditional shilling attacks (handcrafted fake profiles) and adversarial examples (perturbed inputs) in the context of recommendation systems.
• To evaluate the vulnerability of low-rank recommendation models to end-to-end, learned adversarial attacks under realistic assumptions of adversary knowledge.

Proposed method

• Formulates adversarial recommendation as a repeated general-sum game between an oblivious recommender $ R $ and an adversarial fake user generator $ A $, with two objectives: (G1) realism of fake user ratings, and (G2) optimization of attack intent.
• Uses Generative Adversarial Networks (GANs) to learn the rating distribution of real users and generate realistic fake user profiles that mimic this distribution.
• Employs 0-th order optimization to update fake user profiles without gradient access to the recommender model, enabling gradient-free attack optimization.
• Integrates a loss function that balances realism (e.g., similarity in rating distribution and eigen-spectrum) with adversarial objectives such as minimizing predicted scores for target users or items.
• Assumes the adversary knows the recommender’s model architecture and can retrain it on augmented data, enabling iterative refinement of fake profiles.
• Evaluates the attack by measuring changes in recommendation quality (e.g., top-K performance) after injecting fake users into the training set.

Experimental results

Research questions

• RQ1 Can machine learning models for recommendation be effectively attacked by fake user profiles that are indistinguishable from real users?
• RQ2 To what extent can a GAN-based generator produce fake user profiles that mimic real user rating patterns while achieving specific adversarial goals?
• RQ3 How effective are 0-th order optimization methods in crafting adversarial attacks when the adversary lacks gradient access to the recommender model?
• RQ4 What is the impact of learned fake users on the top-K recommendation quality for targeted users or items?
• RQ5 Can a single strategic manipulation—such as minimizing the predicted score of the highest-rated user—lead to widespread degradation of recommendation quality for a target item?

Key findings

• The proposed framework successfully generates fake user profiles that are statistically indistinguishable from real users in terms of rating distribution and eigen-spectrum, making them highly realistic.
• The attack significantly degrades the top-K recommendation quality for targeted users, with one key finding showing that minimizing the predicted score of the top-rated user for a target item is sufficient to ruin its recommendation performance.
• The method achieves high attack success across multiple attack intents, including promoting or demoting specific items, and works effectively even with limited knowledge of the recommender model.
• Experiments demonstrate that the GAN-based generator outperforms handcrafted fake profiles in both realism and attack effectiveness, confirming the superiority of learned over heuristic approaches.
• The framework reveals that recommendation systems are vulnerable to data-poisoning attacks via realistic, learned fake users, even when the adversary does not have access to the model’s gradients.
• The results suggest that current recommendation systems may be insufficiently robust to data-level adversarial attacks, highlighting the need for adversary-aware defenses.