[Paper Review] Learning to Defer in Congested Systems: The AI-Human Interplay

Thodoris Lykouris, Wentao Weng|arXiv (Cornell University)|Feb 19, 2024

Hate Speech and Cyberbullying Detection4 citations

TL;DR

This paper proposes a novel online learning framework for content moderation that models the human-AI interplay under congestion, where AI decisions to defer posts to human review are constrained by variable human review capacity. The algorithm balances classification accuracy, selective sampling bias, and queueing delays through a contextual queueing model, achieving near-optimal regret bounds in a setting with endogenous feedback delays.

ABSTRACT

High-stakes applications rely on combining Artificial Intelligence (AI) and humans for responsive and reliable decision making. For example, content moderation in social media platforms often employs an AI-human pipeline to promptly remove policy violations without jeopardizing legitimate content. A typical heuristic estimates the risk of incoming content and uses fixed thresholds to decide whether to auto-delete the content (classification) and whether to send it for human review (admission). This approach can be inefficient as it disregards the uncertainty in AI's estimation, the time-varying element of content arrivals and human review capacity, and the selective sampling in the online dataset (humans only review content filtered by the AI). In this paper, we introduce a model to capture such an AI-human interplay. In this model, the AI observes contextual information for incoming jobs, makes classification and admission decisions, and schedules admitted jobs for human review. During these reviews, humans observe a job's true cost and may overturn an erroneous AI classification decision. These reviews also serve as new data to train the AI but are delayed due to congestion in the human review system. The objective is to minimize the costs of eventually misclassified jobs. We propose a near-optimal learning algorithm that carefully balances the classification loss from a selectively sampled dataset, the idiosyncratic loss of non-reviewed jobs, and the delay loss of having congestion in the human review system. To the best of our knowledge, this is the first result for online learning in contextual queueing systems. Moreover, numerical experiments based on online comment datasets show that our algorithm can substantially reduce the number of misclassifications compared to existing content moderation practice.

Motivation & Objective

• Address the inefficiency of fixed-threshold AI-human collaboration in content moderation due to ignored uncertainty, time-varying human capacity, and selective sampling.
• Model the human-AI interplay as a contextual queueing system where feedback delays are endogenously determined by admission and scheduling decisions.
• Develop a learning algorithm that jointly optimizes classification loss, idiosyncratic loss from unreviewed posts, and delay loss from human review congestion.
• Provide theoretical guarantees for regret minimization in a novel online learning setting with endogenous, capacity-constrained feedback delays.
• Extend the learning-to-defer paradigm to include soft capacity constraints and congestion effects, filling a gap in existing literature.

Proposed method

• Formalize the content moderation process as a contextual queueing system with time-varying post types, latent harmfulness costs, and human review queues.
• Model the AI’s decisions as classification (accept/reject), admission (defer to human queue), and scheduling (select which post to review next).
• Use ridge regression with confidence sets to estimate the true harmfulness of posts, incorporating uncertainty via Bayesian-style confidence intervals.
• Define a loss function that combines three components: classification error on non-reviewed posts, idiosyncratic loss from unreviewed but harmful content, and delay cost from queueing congestion.
• Introduce a scheduling policy that prioritizes posts based on expected marginal benefit, balancing urgency and uncertainty in the queue.
• Leverage concentration inequalities and matrix determinant bounds to derive high-probability regret bounds for the learning algorithm.

Experimental results

Research questions

• RQ1 How can AI systems optimally balance classification, admission, and scheduling decisions in content moderation when human review capacity is limited and variable?
• RQ2 What is the impact of endogenous feedback delays—caused by admission and scheduling decisions—on learning performance in human-AI collaboration?
• RQ3 How can the learning algorithm minimize regret when the training data is selectively sampled (only admitted posts are reviewed) and feedback is delayed?
• RQ4 What theoretical guarantees can be provided for online learning in a contextual queueing system with soft capacity constraints?
• RQ5 How does congestion in the human review system affect the long-term performance and reliability of AI models in content moderation?

Key findings

• The proposed algorithm achieves a near-optimal regret bound of $ O(\sqrt{T \log T}) $ in the presence of endogenous feedback delays due to congestion.
• The algorithm effectively mitigates the bias from selective sampling by incorporating uncertainty estimates into admission and scheduling decisions.
• The delay cost component in the loss function significantly improves performance under high congestion, reducing the number of unreviewed harmful posts.
• Theoretical analysis shows that the confidence set construction via ridge regression ensures robustness to model misspecification and noisy feedback.
• Empirical validation demonstrates that the algorithm outperforms fixed-threshold baselines in terms of both classification accuracy and fairness across post types.
• The framework is the first to formalize online learning in contextual queueing systems, offering a novel analytical foundation for human-AI collaboration in real-time systems.