[Paper Review] Diverse Misinformation: Impacts of Human Biases on Detection of Deepfakes on Networks

Juniper Lovato, Laurent Hébert‐Dufresne+6 more|arXiv (Cornell University)|Oct 18, 2022

Misinformation and Its Impacts4 citations

TL;DR

This study investigates how human biases and demographic alignment affect susceptibility to deepfakes on social networks, using an observational survey of 2,016 U.S. participants. It finds that users are more accurate at detecting deepfakes when the persona matches their own demographics, and a mathematical model suggests diverse social networks can enable 'herd correction' that reduces collective misinformation susceptibility.

ABSTRACT

Social media platforms often assume that users can self-correct against misinformation. However, social media users are not equally susceptible to all misinformation as their biases influence what types of misinformation might thrive and who might be at risk. We call "diverse misinformation" the complex relationships between human biases and demographics represented in misinformation. To investigate how users' biases impact their susceptibility and their ability to correct each other, we analyze classification of deepfakes as a type of diverse misinformation. We chose deepfakes as a case study for three reasons: 1) their classification as misinformation is more objective; 2) we can control the demographics of the personas presented; 3) deepfakes are a real-world concern with associated harms that must be better understood. Our paper presents an observational survey (N=2,016) where participants are exposed to videos and asked questions about their attributes, not knowing some might be deepfakes. Our analysis investigates the extent to which different users are duped and which perceived demographics of deepfake personas tend to mislead. We find that accuracy varies by demographics, and participants are generally better at classifying videos that match them. We extrapolate from these results to understand the potential population-level impacts of these biases using a mathematical model of the interplay between diverse misinformation and crowd correction. Our model suggests that diverse contacts might provide "herd correction" where friends can protect each other. Altogether, human biases and the attributes of misinformation matter greatly, but having a diverse social group may help reduce susceptibility to misinformation.

Motivation & Objective

• To investigate how individual human biases and demographic alignment influence susceptibility to deepfakes on social media.
• To examine whether users are more accurate at detecting deepfakes when the persona in the video matches their own demographic identity (e.g., race, gender, age).
• To model the population-level impact of diverse misinformation and assess whether diverse social networks can enable collective correction.
• To evaluate the role of perception and bias in misinformation detection, independent of algorithmic model biases in deepfake generation.
• To provide empirical evidence on the interplay between viewer demographics and deepfake detection accuracy using a large-scale survey.

Proposed method

• Conducted an observational survey (N=2,016) using a Qualtrics panel, where participants viewed videos without explicit priming about deepfake detection.
• Used the Matthew’s Correlation Coefficient (MCC) to measure classification accuracy of participants as human classifiers, with values ranging from -1 (complete disagreement) to 1 (perfect agreement) with ground truth.
• Applied bootstrapping (10,000 samples) to compare MCC differences between demographic subgroups and test statistical significance of biases.
• Performed Bayesian logistic regression to model the relationship between demographic matches (age, gender, race) and detection accuracy.
• Developed a mathematical model of misinformation spread and correction in networks to simulate herd correction effects across diverse social contacts.
• Analyzed aggregated, anonymized data from the survey, with full code and data available on GitHub, while protecting participant privacy.

Experimental results

Research questions

• RQ1 How does demographic alignment between a participant and the deepfake persona affect detection accuracy?
• RQ2 Which demographic groups are most or least susceptible to being misled by deepfakes?
• RQ3 To what extent can diverse social networks enable collective correction of misinformation through peer influence?
• RQ4 How do human perception biases—rather than algorithmic model biases—affect deepfake detection outcomes?
• RQ5 What is the population-level impact of diverse misinformation when individuals vary in their susceptibility based on identity features?

Key findings

• Participants were significantly more accurate at detecting deepfakes when the persona’s demographic features (race, gender, age) matched their own, indicating strong viewer bias.
• The Matthew’s Correlation Coefficient (MCC) for detection accuracy varied by demographic group, with higher MCC values observed when participants and personas shared demographic attributes.
• Participants showed higher detection accuracy for deepfakes depicting individuals of the same race, gender, and age group, suggesting that in-group perception enhances critical evaluation.
• The mathematical model demonstrated that diverse social networks can lead to 'herd correction,' where friends with varied demographics collectively reduce misinformation susceptibility.
• Even with balanced deepfake datasets (e.g., Facebook Deepfake Detection Challenge), human perception biases persist and significantly affect detection outcomes.
• The study found no evidence of systematic algorithmic bias in video quality affecting detection, as the observed accuracy differences were driven by viewer perception rather than video fidelity.