[Paper Review] Multi-Armed Bandits with Fairness Constraints for Distributing Resources to Human Teammates

Houston Claure, Yifang Chen+3 more|arXiv (Cornell University)|Jun 30, 2019

Advanced Bandit Algorithms Research61 references4 citations

TL;DR

This paper proposes a fairness-constrained multi-armed bandit algorithm for robots to distribute resources (e.g., attention, parts) among human teammates while learning their skill levels over time. By enforcing minimum selection rates per teammate, the algorithm improves trust—especially among lower-performing individuals—without sacrificing team performance, as shown in a large-scale Tetris user study with 25%, 33%, and 50% fairness constraints.

ABSTRACT

How should a robot that collaborates with multiple people decide upon the distribution of resources (e.g. social attention, or parts needed for an assembly)? People are uniquely attuned to how resources are distributed. A decision to distribute more resources to one team member than another might be perceived as unfair with potentially detrimental effects for trust. We introduce a multi-armed bandit algorithm with fairness constraints, where a robot distributes resources to human teammates of different skill levels. In this problem, the robot does not know the skill level of each human teammate, but learns it by observing their performance over time. We define fairness as a constraint on the minimum rate that each human teammate is selected throughout the task. We provide theoretical guarantees on performance and perform a large-scale user study, where we adjust the level of fairness in our algorithm. Results show that fairness in resource distribution has a significant effect on users' trust in the system.

Motivation & Objective

• To address the challenge of resource distribution in human-robot teams where unequal allocation may undermine trust.
• To formalize fairness as a minimum selection rate constraint for each human teammate in a multi-armed bandit framework.
• To develop a theoretical algorithm that balances learning teammate skill levels with fairness constraints.
• To evaluate the impact of fairness on human perceptions of trust and team performance in collaborative settings.
• To demonstrate that fairness in resource allocation enhances trust without degrading overall team performance.

Proposed method

• Formulates the resource distribution problem as a multi-armed bandit with arms representing human teammates and stochastic rewards reflecting skill levels.
• Proposes a stochastic UCB-based algorithm that ensures each arm is pulled at least at a minimum rate in expectation.
• Introduces a deterministic variant that strictly enforces the minimum pulling rate at all times.
• Defines a new regret metric that accounts for fairness constraints, providing theoretical regret bounds.
• Employs a Tetris-based user study where the algorithm assigns block batches to two players based on their performance and fairness constraints.
• Uses three fairness conditions (25%, 33%, 50%) to vary the minimum allocation rate and assess perception and performance outcomes.

Experimental results

Research questions

• RQ1 How does enforcing a minimum selection rate for each teammate affect users’ perception of fairness and trust in a collaborative system?
• RQ2 Does fairness in resource allocation impact team performance, particularly when higher-performing teammates are selected less frequently?
• RQ3 Can a multi-armed bandit algorithm maintain high team performance while guaranteeing fairness through theoretical constraints?
• RQ4 How do lower-performing teammates perceive fairness when their resource allocation is constrained, compared to higher-performing teammates?
• RQ5 To what extent does perceived fairness mediate trust in autonomous systems that make resource allocation decisions?

Key findings

• Participants in the 33% fairness condition reported significantly higher trust in the system compared to those in the 25% condition, particularly among lower-performing players.
• Despite selecting the stronger player less frequently, team performance—measured by median scores—improved with higher fairness levels, with the 50% condition showing the highest median scores.
• Perceived fairness was significantly lower in the 25% condition due to a larger disparity in turn allocation between strong and weak players, while the 50% condition showed equal participation and higher fairness perception.
• There was no significant decline in overall team performance across fairness conditions, contradicting the hypothesis that fairness reduces performance.
• The study found that fairness perceptions were strongly tied to perceived equity in participation, not just outcome equality, with stronger players not perceiving the system as unfair even when selected less.
• The theoretical regret bounds show that the fairness-constrained algorithm performs comparably to unconstrained UCB, with no worse regret than the oracle under the same constraints.