[Paper Review] Measuring Discrepancies in Airbnb Guest Acceptance Rates Using Anonymized Demographic Data

Siddhartha Basu, Ruthie Berman+7 more|arXiv (Cornell University)|Apr 25, 2022

Sharing Economy and Platforms4 citations

TL;DR

This paper proposes a privacy-preserving system to measure racial experience gaps in Airbnb's guest booking process using anonymized demographic data, applying p-sensitive k-anonymity to prevent re-identification. It demonstrates through simulation that the method achieves precision comparable to non-anonymized data, enabling equitable platform policy evaluation without compromising user privacy.

ABSTRACT

In order to make technological systems and platforms more equitable, organizations must be able to measure the scale of potential inequities as well as the efficacy of proposed solutions. In this paper, we present a system that measures discrepancies in platform user experience that are attributable to perceived race (experience gaps) using anonymized data. This allows for progress to be made in this area while limiting any potential privacy risk. Specifically, the system enforces the privacy model of p-sensitive k-anonymity to conduct measurement without ever storing or having access to a 1:1 mapping between user identifiers and perceived race. We test this system in the context of the Airbnb guest booking experience. Our simulation-based power analysis shows that the system can measure the efficacy of proposed platform-wide interventions with comparable precision to non-anonymized data. Our work establishes that measurement of experience gaps with anonymized data is feasible and can be used to guide the development of policies to promote equitable outcomes for users of Airbnb as well as other technology platforms.

Motivation & Objective

• To develop a method for measuring perceived race-based disparities in Airbnb guest acceptance rates without compromising user privacy.
• To ensure that no 1:1 mapping between user identifiers and perceived race is ever stored or accessed.
• To evaluate the efficacy of platform-wide interventions on equity using anonymized data with precision comparable to non-anonymized approaches.
• To establish a scalable, privacy-preserving framework applicable to other digital platforms facing similar equity challenges.

Proposed method

• The system uses p-sensitive k-anonymity to anonymize user data, ensuring that each user's record is indistinguishable from at least k-1 others with the same perceived race.
• Anonymized demographic data is derived from name-based inference models trained on public data to estimate perceived race.
• A simulation-based power analysis is conducted to assess the statistical precision of detecting differences in guest acceptance rates across perceived racial groups.
• The method avoids storing or accessing direct links between user identifiers and perceived race, minimizing privacy risks.
• The framework is validated through synthetic data simulations to evaluate its ability to detect meaningful differences in acceptance rates.

Experimental results

Research questions

• RQ1 Can experience gaps in Airbnb's guest booking process be measured accurately using anonymized demographic data?
• RQ2 To what extent does the p-sensitive k-anonymity model preserve privacy while maintaining measurement precision?
• RQ3 How does the precision of disparity detection using anonymized data compare to non-anonymized data?
• RQ4 Can this system reliably evaluate the impact of platform-wide equity interventions?

Key findings

• The system achieves measurement precision comparable to non-anonymized data, as confirmed by simulation-based power analysis.
• The p-sensitive k-anonymity model successfully prevents re-identification by ensuring no 1:1 mapping between user IDs and perceived race.
• The method enables detection of statistically significant differences in guest acceptance rates across perceived racial groups with high confidence.
• The framework is scalable and generalizable to other digital platforms seeking to audit and improve equity in user experiences.