[Paper Review] Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits

Jack Bandy|arXiv (Cornell University)|Feb 3, 2021

Ethics and Social Impacts of AI12 references32 citations

TL;DR

The paper conducts a PRISMA-guided scoping review of 500+ papers, synthesizing 62 algorithm audit studies into a four-behavior taxonomy (discrimination, distortion, exploitation, misjudgement) and outlines future audit directions.

ABSTRACT

While algorithm audits are growing rapidly in commonality and public importance, relatively little scholarly work has gone toward synthesizing prior work and strategizing future research in the area. This systematic literature review aims to do just that, following PRISMA guidelines in a review of over 500 English articles that yielded 62 algorithm audit studies. The studies are synthesized and organized primarily by behavior (discrimination, distortion, exploitation, and misjudgement), with codes also provided for domain (e.g. search, vision, advertising, etc.), organization (e.g. Google, Facebook, Amazon, etc.), and audit method (e.g. sock puppet, direct scrape, crowdsourcing, etc.). The review shows how previous audit studies have exposed public-facing algorithms exhibiting problematic behavior, such as search algorithms culpable of distortion and advertising algorithms culpable of discrimination. Based on the studies reviewed, it also suggests some behaviors (e.g. discrimination on the basis of intersectional identities), domains (e.g. advertising algorithms), methods (e.g. code auditing), and organizations (e.g. Twitter, TikTok, LinkedIn) that call for future audit attention. The paper concludes by offering the common ingredients of successful audits, and discussing algorithm auditing in the context of broader research working toward algorithmic justice.

Motivation & Objective

• Map existing algorithm audit studies to identify common harms.
• Develop a taxonomy of problematic machine behaviors.
• Analyze domains, organizations, and audit methods used in audits.
• Identify gaps and propose future audit directions for algorithmic justice.

Proposed method

• screened 500+ papers using PRISMA guidelines to identify algorithm audits.
• conducted thematic analysis with coding by behavior, domain, organization, and audit method.
• used inductive thematic analysis with two coders and Cohen’s kappa progression.
• defined four problematic behaviors: discrimination, distortion, exploitation, misjudgement.
• supplemented keyword-driven identification with Scopus and additional sources to reduce recency bias.

Experimental results

Research questions

• RQ1 RQ1: What kinds of problematic machine behavior have previous algorithm audits exposed?
• RQ2 RQ2: What remains for future algorithm audits to examine the problematic ways that algorithms exercise power in society?

Key findings

• 62 studies analyzed; most focused on discrimination (21) or distortion (29).
• Audits covered search (25), advertising (12), and recommendation (8).
• Identified four main behaviors: discrimination, distortion, exploitation, misjudgement.
• Advertising and vision domains show notable discrimination and distortion patterns.
• Code auditing is under-explored; limited coverage of platforms like Twitter, LinkedIn, TikTok.
• Calls for auditing intersectional discrimination and framing audits within broader algorithmic justice.