[Paper Review] Algorithms that "Don't See Color": Comparing Biases in Lookalike and Special Ad Audiences

Piotr Sapieżyński, Avijit Ghosh+3 more|arXiv (Cornell University)|Dec 16, 2019

Ethics and Social Impacts of AI26 references10 citations

TL;DR

This study evaluates Facebook's Special Ad Audiences algorithm—designed to exclude demographic features like gender, age, and race—to test whether removing such inputs eliminates bias in ad targeting. Despite the removal of sensitive attributes, the algorithm still produces highly skewed audiences nearly identical to standard Lookalike Audiences, demonstrating that bias persists through correlated features and content-based delivery mechanisms.

ABSTRACT

Researchers and journalists have repeatedly shown that algorithms commonly used in domains such as credit, employment, healthcare, or criminal justice can have discriminatory effects. Some organizations have tried to mitigate these effects by simply removing sensitive features from an algorithm's inputs. In this paper, we explore the limits of this approach using a unique opportunity. In 2019, Facebook agreed to settle a lawsuit by removing certain sensitive features from inputs of an algorithm that identifies users similar to those provided by an advertiser for ad targeting, making both the modified and unmodified versions of the algorithm available to advertisers. We develop methodologies to measure biases along the lines of gender, age, and race in the audiences created by this modified algorithm, relative to the unmodified one. Our results provide experimental proof that merely removing demographic features from a real-world algorithmic system's inputs can fail to prevent biased outputs. As a result, organizations using algorithms to help mediate access to important life opportunities should consider other approaches to mitigating discriminatory effects.

Motivation & Objective

• To evaluate whether removing sensitive demographic features (e.g., gender, age, race) from Facebook’s Lookalike Audience algorithm reduces demographic skew in ad targeting.
• To investigate whether Facebook’s Special Ad Audiences tool—introduced as a compliance measure after a lawsuit—effectively mitigates bias compared to standard Lookalike Audiences.
• To examine how content-based ad delivery mechanisms and source audience composition contribute to persistent demographic skew even when protected attributes are excluded.
• To assess the real-world implications of algorithmic bias in high-stakes domains like housing, employment, and credit advertising.
• To develop a methodology for measuring and comparing bias across algorithmic targeting systems in real-world, large-scale platforms.

Proposed method

• Constructed source audiences with known demographic imbalances (e.g., all female, predominantly male, or diverse) to test audience skew in Lookalike and Special Ad Audiences.
• Deployed identical real-world ads (e.g., job listings) through both audience types to measure delivery distribution across gender, age, and race groups.
• Used statistical analysis to compare delivery distributions between Lookalike and Special Ad Audiences, testing for significant differences in demographic representation.
• Evaluated content-based skew by testing different ad copy (e.g., for supermarket jobs vs. AI jobs) on the same target audience to isolate the impact of ad content on delivery bias.
• Analyzed real-world ad delivery patterns using a Special Ad Audience derived from Facebook employees to assess real-world applicability and skew.
• Employed a mixed-methods approach combining controlled experiments with observational analysis of platform behavior to assess systemic bias.

Experimental results

Research questions

• RQ1 To what extent does removing demographic features from Facebook’s Lookalike Audience algorithm reduce demographic skew in target audiences?
• RQ2 How do the demographic compositions of Special Ad Audiences compare to those of standard Lookalike Audiences when using the same source audience?
• RQ3 To what extent do content-based delivery mechanisms in Facebook’s ad system contribute to demographic skew, even when target audiences are balanced?
• RQ4 How does the composition of the source audience influence the skew in the resulting target audience, regardless of whether demographic features are excluded?
• RQ5 What are the legal and ethical implications of using algorithmic targeting tools that produce biased outcomes despite the removal of sensitive attributes?

Key findings

• When targeting a source audience that was 100% female, the Lookalike Audience delivered the ad to 96.1% women, while the Special Ad Audience delivered it to 91.2% women—differences that were not statistically significant.
• The Special Ad Audience tool did not meaningfully reduce skew along gender, age, or racial lines compared to standard Lookalike Audiences when using the same source audience.
• A Special Ad Audience created from Facebook employees (predominantly male and aged 25–34) resulted in a target audience that was heavily skewed toward 25–34-year-old men, indicating persistent demographic bias.
• Content-based delivery mechanisms caused significant skew: ads for supermarket jobs were disproportionately shown to middle-aged women, while AI job ads were shown more to younger men, even when targeting the same balanced audience.
• The study confirms that Facebook’s ad delivery optimization system can amplify bias through content relevance estimation, even when demographic features are excluded from the targeting algorithm.
• The results suggest that removing sensitive features from algorithm inputs is insufficient to prevent discriminatory outcomes in real-world algorithmic systems, particularly in complex socio-technical environments.