[Paper Review] Task Recommendation in Crowdsourcing Based on Learning Preferences and Reliabilities

Qiyu Kang, Wee Peng Tay|arXiv (Cornell University)|Jul 27, 2018

Mobile Crowdsensing and Crowdsourcing33 references4 citations

TL;DR

This paper proposes a multi-armed bandit (MAB) framework for task recommendation in crowdsourcing that learns worker preferences and reliabilities using gold tasks—tasks with known answers but no rewards. The model achieves optimal regret of $O(\sqrt{n})$ and introduces three strategies (GR, UR, $\epsilon$-first) that are order-optimal, with simulations showing GR outperforms uniform random exploration in cumulative reward and regret.

ABSTRACT

Workers participating in a crowdsourcing platform can have a wide range of abilities and interests. An important problem in crowdsourcing is the task recommendation problem, in which tasks that best match a particular worker's preferences and reliabilities are recommended to that worker. A task recommendation scheme that assigns tasks more likely to be accepted by a worker who is more likely to complete it reliably results in better performance for the task requester. Without prior information about a worker, his preferences and reliabilities need to be learned over time. In this paper, we propose a multi-armed bandit (MAB) framework to learn a worker's preferences and his reliabilities for different categories of tasks. However, unlike the classical MAB problem, the reward from the worker's completion of a task is unobservable. We therefore include the use of gold tasks (i.e., tasks whose solutions are known \emph{a priori} and which do not produce any rewards) in our task recommendation procedure. Our model could be viewed as a new variant of MAB, in which the random rewards can only be observed at those time steps where gold tasks are used, and the accuracy of estimating the expected reward of recommending a task to a worker depends on the number of gold tasks used. We show that the optimal regret is $O(\sqrt{n})$, where $n$ is the number of tasks recommended to the worker. We develop three task recommendation strategies to determine the number of gold tasks for different task categories, and show that they are order optimal. Simulations verify the efficiency of our approaches.

Motivation & Objective

• To address the challenge of recommending tasks in crowdsourcing platforms that match worker preferences and reliabilities to improve task completion quality and acceptance.
• To model the task recommendation problem as a multi-armed bandit (MAB) setting with unobservable rewards due to lack of immediate feedback on task quality.
• To develop a learning mechanism that uses gold tasks—known-solution tasks without rewards—to estimate worker preferences and reliabilities over time.
• To design and evaluate task recommendation strategies that balance exploration (using gold tasks) and exploitation (assigning non-gold tasks) efficiently.
• To demonstrate that the proposed strategies achieve optimal regret order $O(\sqrt{n})$ and outperform baseline methods in cumulative reward and regret.

Proposed method

• Formulates task recommendation as a multi-armed bandit problem where arms correspond to task categories, and rewards are unobservable.
• Introduces gold tasks—tasks with known solutions but no reward—to serve as feedback mechanisms for estimating worker preferences and reliabilities.
• Uses empirical mean estimation of task category performance based on gold task outcomes to infer expected rewards, with estimation accuracy dependent on the number of gold tasks used.
• Proposes three strategies: Uniform Random (UR), Greedy with Reward (GR), and $\epsilon$-first, each determining the number of gold tasks per category based on exploration-exploitation trade-offs.
• Analyzes regret using theoretical bounds and shows that the optimal regret order is $O(\sqrt{n})$, where $n$ is the number of recommended tasks.
• Employs simulation experiments with varying parameters ($K$, $\alpha$, $\min_{k\neq*}\Delta_k$) to evaluate performance across different settings and strategy variants.

Experimental results

Research questions

• RQ1 Can a multi-armed bandit framework effectively learn worker preferences and reliabilities in crowdsourcing when task rewards are unobservable?
• RQ2 How can gold tasks be strategically used to estimate worker performance and guide task recommendations without affecting reward collection?
• RQ3 What is the optimal regret order for task recommendation in this unobservable reward setting, and can it be achieved by practical strategies?
• RQ4 How do different exploration strategies—uniform random, greedy, and $\epsilon$-first—compare in terms of cumulative reward and regret under varying conditions?
• RQ5 Does learning both preferences and reliabilities simultaneously lead to better performance than learning only one aspect?

Key findings

• The optimal regret order for the proposed task recommendation framework is $O(\sqrt{n})$, where $n$ is the number of recommended tasks, establishing a theoretical performance bound.
• The Greedy with Reward (GR) strategy consistently outperforms Uniform Random (UR) by reducing estimation variance and improving cumulative reward through smarter gold task allocation.
• A moderate value of the exploration parameter $\alpha$ yields optimal performance in both GR and UR strategies, as too high or too low $\alpha$ increases regret.
• When $\min_{k\neq*}\Delta_k$ is small (i.e., the best task category is hard to distinguish from others), the average regret for GR increases, indicating higher difficulty in identification.
• Learning only preferences or only reliabilities results in significantly lower cumulative rewards compared to learning both, with the performance gap widening over time.