[Paper Review] An Information Theoretic Framework For Designing Information Elicitation Mechanisms That Reward Truth-telling

Yuqing Kong, Grant Schoenebeck|arXiv (Cornell University)|May 3, 2016

Mobile Crowdsensing and Crowdsourcing26 references12 citations

TL;DR

This paper introduces the Mutual Information Paradigm, an information-theoretic framework that designs truthful, detail-free, and minimal information elicitation mechanisms by rewarding agents based on the mutual information between their signals and peers' signals. The key contribution is a family of mechanisms—f-mutual information and Bregman mutual information—where truth-telling is a dominant strategy and strictly dominates all non-truthful strategies, even under arbitrary peer behavior.

ABSTRACT

In the setting where information cannot be verified, we propose a simple yet powerful information theoretical framework---the Mutual Information Paradigm---for information elicitation mechanisms. Our framework pays every agent a measure of mutual information between her signal and a peer's signal. We require that the mutual information measurement has the key property that any "data processing" on the two random variables will decrease the mutual information between them. We identify such information measures that generalize Shannon mutual information. Our Mutual Information Paradigm overcomes the two main challenges in information elicitation without verification: (1) how to incentivize effort and avoid agents colluding to report random or identical responses (2) how to motivate agents who believe they are in the minority to report truthfully. Aided by the information measures we found, (1) we use the paradigm to design a family of novel mechanisms where truth-telling is a dominant strategy and any other strategy will decrease every agent's expected payment (in the multi-question, detail free, minimal setting where the number of questions is large); (2) we show the versatility of our framework by providing a unified theoretical understanding of existing mechanisms---Peer Prediction [Miller 2005], Bayesian Truth Serum [Prelec 2004], and Dasgupta and Ghosh [2013]---by mapping them into our framework such that theoretical results of those existing mechanisms can be reconstructed easily. We also give an impossibility result which illustrates, in a certain sense, the the optimality of our framework.

Motivation & Objective

• To address the challenge of incentivizing truthful reporting in information elicitation without verifiable ground truth.
• To overcome the dual challenge of discouraging collusion (e.g., reporting identical or random responses) and motivating minority-reporting agents to be truthful.
• To design mechanisms that are truthful, focal, detail-free, minimal, and dominantly truthful—offering stronger equilibrium guarantees than Bayesian Nash equilibrium.
• To unify and re-interpret existing mechanisms (e.g., Peer Prediction, Bayesian Truth Serum) within a single information-theoretic framework.
• To establish theoretical optimality of the framework via impossibility results under minimal assumptions.

Proposed method

• Proposes the Mutual Information Paradigm, where agents are paid based on the mutual information between their private signal and a peer’s signal.
• Identifies two families of information measures—f-mutual information and Bregman mutual information—that generalize Shannon mutual information and satisfy the data processing inequality.
• Uses the data processing inequality to ensure that any non-truthful strategy reduces mutual information, thus reducing expected payment.
• Applies the framework to design novel mechanisms in both multi-question and single-question settings with dominant truth-telling incentives.
• Demonstrates that the log scoring rule is an unbiased estimator of Shannon mutual information in the single-question setting.
• Employs indistinguishable scenarios and coupling arguments to prove impossibility results, showing the optimality of the framework under symmetric priors and permutation invariance.

Experimental results

Research questions

• RQ1 Can a general information-theoretic framework be constructed to design truthful, detail-free, and minimal information elicitation mechanisms?
• RQ2 How can mutual information be generalized to ensure that non-truthful strategies always reduce expected payment?
• RQ3 Can existing mechanisms like Peer Prediction and Bayesian Truth Serum be unified and reinterpreted through this framework?
• RQ4 Is it possible to design a mechanism where truth-telling is a dominant strategy rather than just a Bayesian Nash equilibrium?
• RQ5 What are the fundamental limitations of truthful, detail-free mechanisms under symmetric priors and permutation invariance?

Key findings

• The f-mutual information and Bregman mutual information mechanisms ensure that truth-telling is a dominant strategy in the multi-question setting when the number of questions is large.
• In the f-mutual information mechanism, any deviation from truth-telling strictly reduces every agent’s expected payment, regardless of other agents’ strategies, establishing both dominance and focal properties.
• The framework successfully reconstructs and unifies existing mechanisms: Peer Prediction, Bayesian Truth Serum, and Dasgupta and Ghosh (2013), by mapping them into the mutual information paradigm.
• The log scoring rule is shown to be an unbiased estimator of Shannon mutual information in the single-question setting.
• An impossibility result proves that no truthful, detail-free mechanism can strictly outperform permutation strategies in symmetric settings unless it leverages the data processing inequality—highlighting the optimality of the proposed framework.
• The framework demonstrates that truth-telling is strictly better than any non-permutation strategy in settings with effort costs, under the assumption of symmetric priors.