[Paper Review] WES: Agent-based User Interaction Simulation on Real Infrastructure

John Ahlgren, Maria Eugenia Berezin+10 more|arXiv (Cornell University)|Apr 11, 2020

Software Engineering Research43 references4 citations

TL;DR

This paper introduces Web-Enabled Simulation (WES), a novel research agenda that uses real platform infrastructure to simulate user communities via agent-based bots, enabling scalable, isolated testing of social interactions. The approach enables Social Testing and Automated Mechanism Design, improving reliability, integrity, and privacy in large-scale systems like Facebook’s platform with hundreds of millions of code lines.

ABSTRACT

We introduce the Web-Enabled Simulation (WES) research agenda, and describe FACEBOOK's WW system. We describe the application of WW to reliability, integrity and privacy at FACEBOOK , where it is used to simulate social media interactions on an infrastructure consisting of hundreds of millions of lines of code. The WES agenda draws on research from many areas of study, including Search Based Software Engineering, Machine Learning, Programming Languages, Multi Agent Systems, Graph Theory, Game AI, and AI Assisted Game Play. We conclude with a set of open problems and research challenges to motivate wider investigation.

Motivation & Objective

• To address the growing challenge of social bugs arising from complex user interactions in large-scale software platforms.
• To develop a simulation framework that leverages real platform infrastructure rather than abstract models, enhancing realism and fidelity.
• To introduce new testing paradigms—Social Testing and Automated Mechanism Design—targeting emergent behaviors and systemic improvements.
• To motivate interdisciplinary research at the intersection of software engineering, AI, game theory, and multi-agent systems for community-level system behavior analysis.

Proposed method

• Employs a multi-agent system where each agent is a bot simulating user behavior, either rule-based or learned via supervised or reinforcement learning.
• Executes simulations on the actual production infrastructure (e.g., Facebook’s codebase), ensuring platform realism despite isolation from live users.
• Uses synthetic graph generation to create representative, isolated social networks for writer bots, avoiding unintended interactions with real users.
• Applies Search Based Software Engineering (SBSE) and Mechanism Design to automate the discovery of system improvements through co-evolutionary optimization.
• Integrates game-theoretic modeling to analyze WES executions as strategic interactions, enabling formal analysis of emergent behaviors.
• Supports counter-factual simulations by allowing bots to explore non-actual or prohibited behaviors in a safe, isolated environment.

Experimental results

Research questions

• RQ1 How can real-world software infrastructure be leveraged to simulate complex user interactions without relying on abstract models?
• RQ2 What new testing abstractions emerge when focusing on user-to-user interactions rather than system-level behavior?
• RQ3 To what extent can automated mechanism design improve platform properties like integrity and privacy through simulation-driven optimization?
• RQ4 How can synthetic graph generation enable safe, large-scale simulation of writer bots without compromising real user isolation?
• RQ5 What role does causality play in translating simulation insights into actionable, real-world system improvements?

Key findings

• The WES approach enables high-fidelity simulation of social media interactions by executing bots on the real platform infrastructure, preserving system-level behaviors and edge cases.
• Social Testing emerges as a new abstraction layer above system-level testing, enabling the detection of emergent social bugs that traditional testing misses.
• Automated Mechanism Design, combining SBSE and mechanism design, successfully identifies improvements to platform policies and interactions in simulation.
• Synthetic graph generation allows for the deployment of fully isolated writer bots that can explore arbitrary behaviors without violating isolation constraints.
• The integration of game theory and causal reasoning in WES simulations enables formal analysis and actionable insights for real-world system hardening.
• The approach is extensible beyond social media, with potential applications in simulating developer communities and software ecosystems using mining techniques from MSR.