[Paper Review] Participatory Budgeting Design for the Real World

Roy Fairstein, Gerdus Benadè+1 more|arXiv (Cornell University)|Feb 26, 2023

Economic and Environmental ValuationEconomics, Econometrics and Finance3 citations

TL;DR

This study evaluates participatory budgeting (PB) design by testing voting formats and aggregation methods in a controlled experiment with 1,800 participants. It finds that k-approval voting delivers the best user experience, while the Method of Equal Shares (ES) produces stable, format-insensitive outcomes even with low participation, outperforming greedy aggregation in robustness and fairness.

ABSTRACT

Participatory budgeting engages the public in the process of allocating public money to different types of projects. PB designs differ in how voters are asked to express their preferences over candidate projects and how these preferences are aggregated to determine which projects to fund. This paper studies two fundamental questions in PB design. Which voting format and aggregation method to use, and how to evaluate the outcomes of these design decisions? We conduct an extensive empirical study in which 1 800 participants vote in four participatory budgeting elections in a controlled setting to evaluate the practical effects of the choice of voting format and aggregation rule. We find that k-approval leads to the best user experience. With respect to the aggregation rule, greedy aggregation leads to outcomes that are highly sensitive to the input format used and the fraction of the population that participates. The method of equal shares, in contrast, leads to outcomes that are not sensitive to the type of voting format used, and these outcomes are remarkably stable even when the majority of the population does not participate in the election. These results carry valuable insights for PB practitioners and social choice researchers.

Motivation & Objective

• To evaluate how different voting formats and aggregation rules affect user experience and election outcomes in participatory budgeting (PB).
• To identify the most effective voting format that balances usability, preference expression, and accessibility for diverse voters.
• To assess the robustness and fairness of aggregation methods—particularly greedy aggregation versus the Method of Equal Shares (ES)—under varying participation levels.
• To provide evidence-based design recommendations for PB practitioners aiming to increase participation and ensure equitable outcomes.
• To explore whether input format choice significantly distorts outcomes when using different aggregation rules, especially under low participation.

Proposed method

• Conducted a controlled lab experiment with 1,800 participants recruited via Amazon Mechanical Turk across four distinct PB elections.
• Assigned each participant one of six voting formats: k-approval, points, knapsack, ranking, value-for-money, and utility reporting.
• Measured user experience through time-on-task, self-reported usability, and perceived accuracy of preference expression.
• Used the same set of candidate projects and voter preferences across formats to ensure comparability, treating aggregated votes as proxies for voter utilities.
• Applied two aggregation rules—greedy (standard in practice) and Method of Equal Shares (ES)—to compare outcomes across formats and participation levels.
• Evaluated stability by sampling subsets of voters (25–50%) and measuring whether funded project sets changed under each aggregation method.

Figure 1 : The GUI shown to participants using the Knap input format (center), showing the city map (right) and budget (left)

Experimental results

Research questions

• RQ1 Which voting format—k-approval, points, knapsack, ranking, value-for-money, or utility reporting—leads to the best user experience in terms of ease of use, learning time, and perceived accuracy of preference expression?
• RQ2 How do different aggregation rules—greedy vs. Method of Equal Shares (ES)—affect the stability and fairness of PB outcomes when participation is low?
• RQ3 To what extent is the outcome of PB elections sensitive to the choice of voting format when using greedy aggregation versus ES?
• RQ4 How does the social welfare of the elected projects compare across input formats and aggregation methods?
• RQ5 How stable are the funded project sets when only a fraction of voters participate, and does this stability differ between greedy and ES aggregation?

Key findings

• k-approval voting led to the best user experience: participants spent the least time learning and using it, and reported it as the most intuitive and accurate for expressing preferences.
• Despite capturing less preference information, k-approval was perceived as more accurate than richer formats like rankings or utility reporting, suggesting simplicity enhances perceived expressiveness.
• Greedy aggregation produced outcomes highly sensitive to both input format and participation levels—changing results significantly when participation dropped below 100%.
• The Method of Equal Shares (ES) produced nearly identical project sets regardless of input format, demonstrating strong format-invariance and robustness.
• ES maintained stable outcomes even when only 25–50% of voters participated, with minimal changes to the funded project set, unlike greedy aggregation which was highly sensitive to low participation.
• While both greedy and ES achieved comparable social welfare, ES offered superior stability and insensitivity to design choices, making it more suitable for real-world PB with low participation.

Figure 2 : Average time ( $s$ ) to complete each stage.