[Paper Review] A comparison of gerrymandering metrics

Gregory S. Warrington|arXiv (Cornell University)|May 31, 2018

Electoral Systems and Political Participation20 references2 citations

TL;DR

This paper evaluates fourteen vote-based metrics for detecting partisan gerrymandering by testing them on a set of predefined fair and unfair hypothetical elections. The declination emerges as the most accurate measure, minimizing both false positives and false negatives in identifying partisan gerrymandering.

ABSTRACT

We compare and contrast fourteen measures that have been proposed for the purpose of quantifying partisan gerrymandering. We consider measures that, rather than examining the shapes of districts, utilize only the partisan vote distribution among districts. The measures considered are two versions of partisan bias; the efficiency gap and several of its variants; the mean-median difference and the equal vote weight standard; the declination and one variant; and the lopsided-means test. Our primary means of evaluating these measures is a suite of hypothetical elections we classify from the start as fair or unfair. We conclude that the declination is the most successful measure in terms of avoiding false positives and false negatives on the elections considered. We include in an appendix the most extreme outliers for each measure among historical congressional and state legislative elections.

Motivation & Objective

• To evaluate the effectiveness of fourteen vote-based metrics in detecting partisan gerrymandering.
• To assess how well these metrics distinguish between fair and unfair redistricting plans using controlled hypothetical scenarios.
• To identify which metric best avoids false positives and false negatives in detecting partisan bias.

Proposed method

• The study constructs a suite of hypothetical elections explicitly classified as fair or unfair to serve as test cases.
• It applies fourteen distinct gerrymandering metrics—such as the efficiency gap, mean-median difference, declination, and others—to each hypothetical election.
• Each metric is evaluated based on its ability to correctly classify the fairness of the redistricting plan.
• The analysis compares metric performance across the hypothetical scenarios, focusing on accuracy in detecting partisan bias.
• Extreme outliers for each metric are compiled in an appendix using historical congressional and state legislative election data.

Experimental results

Research questions

• RQ1 Which gerrymandering metric most accurately identifies unfair redistricting plans in hypothetical elections?
• RQ2 How do different metrics perform in minimizing false positives and false negatives when evaluating partisan gerrymandering?
• RQ3 Which metric demonstrates the most consistent performance across a diverse set of fair and unfair hypothetical districting plans?

Key findings

• The declination is the most effective metric in minimizing both false positives and false negatives across the tested hypothetical elections.
• The efficiency gap and its variants show strong performance but are outperformed by the declination in accuracy.
• The mean-median difference and equal vote weight standard exhibit notable limitations in distinguishing fair from unfair gerrymandering.
• The lopsided-means test performs poorly compared to other metrics in the evaluation framework.
• The appendix identifies the most extreme outliers for each metric in historical congressional and state legislative elections, highlighting their real-world variability.