[Paper Review] Multi-Armed Bandit Mechanisms for Multi-Slot Sponsored Search Auctions

Akash Das Sarma, Sujit Gujar+1 more|arXiv (Cornell University)|Jan 9, 2010

Advanced Bandit Algorithms Research7 references3 citations

TL;DR

This paper develops truthful multi-armed bandit (MAB) mechanisms for multi-slot sponsored search auctions where click-through rates (CTRs) are initially unknown. It establishes that under unconstrained CTRs, truthful mechanisms incur O(T) regret, but under structured CTRs—such as click precedence, coarse pre-estimates, or separable CTRs—weak pointwise monotonicity and weakly separated allocations are necessary and sufficient for truthfulness, with regret bounded by O(T^{2/3}).

ABSTRACT

In pay-per click sponsored search auctions which are currently extensively used by search engines, the auction for a keyword involves a certain number of advertisers (say k) competing for available slots (say m) to display their ads. This auction is typically conducted for a number of rounds (say T). There are click probabilities mu_ij associated with each agent-slot pairs. The goal of the search engine is to maximize social welfare of the advertisers, that is, the sum of values of the advertisers. The search engine does not know the true values advertisers have for a click to their respective ads and also does not know the click probabilities mu_ij s. A key problem for the search engine therefore is to learn these click probabilities during the T rounds of the auction and also to ensure that the auction mechanism is truthful. Mechanisms for addressing such learning and incentives issues have recently been introduced and are aptly referred to as multi-armed-bandit (MAB) mechanisms. When m = 1, characterizations for truthful MAB mechanisms are available in the literature and it has been shown that the regret for such mechanisms will be O(T^{2/3}). In this paper, we seek to derive a characterization in the realistic but non-trivial general case when m > 1 and obtain several interesting results.

Motivation & Objective

• To design truthful multi-armed bandit mechanisms for multi-slot sponsored search auctions where click-through rates (CTRs) are initially unknown.
• To characterize conditions under which MAB mechanisms remain truthful in the presence of strategic bidders and incomplete CTR information.
• To analyze regret bounds for truthful mechanisms under various CTR assumptions, including unconstrained, precedence-ordered, pre-estimated, and separable CTRs.
• To extend prior single-slot MAB mechanism results to the more realistic multi-slot setting with multiple competing advertisers and slots.
• To evaluate the performance of a proposed truthful MAB mechanism via simulation, focusing on average and worst-case regret.

Proposed method

• Proposes a stochastic multi-round auction model where advertisers bid their true valuations and the search engine learns CTRs through repeated auctions over T rounds.
• Introduces the concept of weak pointwise monotonicity and weakly separated allocations as key conditions for truthfulness in MAB mechanisms under structured CTR assumptions.
• Derives a payment rule based on expected utility maximization using estimated CTRs (μ′), ensuring truthful bidding in expectation.
• Uses a round-robin allocation strategy during an initial exploration phase (T^{2/3} rounds) to estimate CTRs before transitioning to exploitation.
• Employs regret analysis comparing actual social welfare to the optimal welfare achievable with full CTR knowledge.
• Conducts simulations with k=4 advertisers and m=2 slots, generating 100T random instances to estimate average and worst-case regret.

Experimental results

Research questions

• RQ1 What conditions on allocation rules ensure truthfulness in multi-slot sponsored search auctions when CTRs are unknown?
• RQ2 How does the regret of truthful MAB mechanisms scale with the number of auction rounds T under unconstrained CTRs?
• RQ3 Can weaker monotonicity conditions (e.g., weak pointwise monotonicity) suffice for truthfulness when CTRs satisfy structural properties like precedence or separability?
• RQ4 What is the achievable regret bound for truthful MAB mechanisms in multi-slot settings under realistic CTR assumptions?
• RQ5 How do the average and worst-case regret behaviors of a truthful MAB mechanism compare to theoretical bounds in simulation?

Key findings

• Under unconstrained CTRs, any truthful MAB mechanism must satisfy strong pointwise monotonicity, and such mechanisms incur O(T) regret.
• When CTRs satisfy a click precedence property, weak pointwise monotonicity is necessary and sufficient for truthfulness in MAB mechanisms.
• With a coarse pre-estimate of CTRs, weak pointwise monotonicity and weakly separated allocations are both necessary and sufficient for truthful MAB mechanisms.
• For separable CTRs (μ_ij = α_i * β_j), the paper provides a characterization of MAB mechanisms that are truthful in expectation.
• Simulation results show that the worst-case regret scales as O(T^{2/3}), closely approximated by (17/3)T^{2/3}, and average-case regret is bounded by (1/3)T^{2/3}, confirming the theoretical bound.
• The experimental evaluation confirms that the proposed truthful MAB mechanism achieves regret within the theoretical O(T^{2/3}) bound, both in average and worst-case scenarios.