[Paper Review] Online Prediction with Selfish Experts

Tim Roughgarden, Okke Schrijvers|arXiv (Cornell University)|Feb 13, 2017

Advanced Bandit Algorithms Research3 citations

TL;DR

This paper introduces a formal model of online prediction with selfish experts who aim to maximize their credibility, showing that incentive-compatible (IC) algorithms—designed using proper scoring rules—are essential for minimizing regret. It proves that with absolute loss, no randomized algorithm (IC or not) can achieve asymptotically vanishing regret, establishing a fundamental separation from the honest-expert setting.

ABSTRACT

We consider the problem of binary prediction with expert advice in settings where experts have agency and seek to maximize their credibility. This paper makes three main contributions. First, it defines a model to reason formally about settings with selfish experts, and demonstrates that "incentive compatible" (IC) algorithms are closely related to the design of proper scoring rules. Designing a good IC algorithm is easy if the designer's loss function is quadratic, but for other loss functions, novel techniques are required. Second, we design IC algorithms with good performance guarantees for the absolute loss function. Third, we give a formal separation between the power of online prediction with selfish experts and online prediction with honest experts by proving lower bounds for both IC and non-IC algorithms. In particular, with selfish experts and the absolute loss function, there is no (randomized) algorithm for online prediction-IC or otherwise-with asymptotically vanishing regret.

Motivation & Objective

• To formalize a model of online prediction where experts act selfishly to maximize their perceived credibility, rather than reporting truthfully.
• To investigate how incentive compatibility (IC) affects performance guarantees in online prediction, especially under non-quadratic loss functions.
• To establish a formal separation between the power of online prediction with selfish experts versus honest experts, particularly under the absolute loss function.
• To design IC algorithms with provable performance guarantees for the absolute loss function, where standard methods fail due to strategic behavior.
• To prove tight lower bounds on regret for both IC and non-IC algorithms, demonstrating that no algorithm can achieve vanishing regret under absolute loss with selfish experts.

Proposed method

• Proposes a novel model of online prediction where experts report beliefs strategically to maximize reputation, not truthfulness.
• Defines incentive compatibility (IC) as a property ensuring experts report their true beliefs, linking it formally to proper scoring rules.
• Designs IC algorithms using proper scoring rules (e.g., Brier, spherical, Beta family) to elicit truthful beliefs from experts.
• Applies randomized weighted majority (RWM) with expert weights updated via IC-compliant scoring rules, rather than standard loss-based updates.
• Uses a hidden Markov model (HMM) with two states to simulate expert behavior, where beliefs shift between high and low confidence states.
• Employs theoretical analysis and simulations to compare IC algorithms against standard weight-update rules, measuring regret relative to the best expert in hindsight.

Experimental results

Research questions

• RQ1 Can online prediction algorithms be designed to be incentive-compatible when experts are selfish and seek to maximize their credibility?
• RQ2 How does the performance of IC algorithms compare to standard online learning algorithms (e.g., RWM with standard loss updates) when experts act strategically?
• RQ3 What are the fundamental limits of online prediction with selfish experts under the absolute loss function?
• RQ4 Is there a provable separation in performance between online prediction with selfish experts and with honest experts?
• RQ5 Can IC algorithms achieve sublinear regret under absolute loss, or are there inherent impossibility results?

Key findings

• For the absolute loss function, no (randomized) online prediction algorithm—IC or not—can achieve asymptotically vanishing regret, establishing a fundamental impossibility.
• IC algorithms based on proper scoring rules (e.g., Brier, spherical, Beta) consistently outperform standard RWM with non-IC updates, achieving regret below 1.02 at T=10,000 and approaching 1.003 at T=200,000.
• The performance gap between IC and non-IC algorithms is stark: the standard RWM algorithm maintains regret around 1.14 at T=200,000, while IC methods approach the performance of the best expert.
• The lower bound from Lemma 20, $2 + rac{1}{ig floor rac{1}{ heta} ig floor}$, is nearly tight in simulations, with empirical regret closely tracking $2 + heta$.
• Simulations show that IC methods perform comparably across different scoring rules (Brier, spherical, Beta), indicating that truthful belief elicitation is more critical than the specific weight-update rule.
• The greedy lower bound instance closely matches the theoretical lower bound, suggesting that the analysis in Lemma 20 is nearly tight, though stronger bounds may exist with refined scoring rule analysis.