[Paper Review] Online Inference for Advertising Auctions

Caio Waisman, Harikesh S. Nair+2 more|arXiv (Cornell University)|Aug 22, 2019

Advanced Bandit Algorithms Research4 citations

TL;DR

This paper proposes a modified Thompson Sampling algorithm that simultaneously optimizes bidding policies and estimates ad exposure effects in real-time advertising auctions by modeling the problem as a multi-armed bandit. It achieves both objectives with lower economic loss than conventional methods, aligning learning and causal inference through the auction's economic structure.

ABSTRACT

Advertisers that engage in real-time bidding (RTB) to display their ads commonly have two goals: learning their optimal bidding policy and estimating the expected effect of exposing users to their ads. Typical strategies to accomplish one of these goals tend to ignore the other, creating an apparent tension between the two. This paper exploits the economic structure of the bid optimization problem faced by advertisers to show that these two objectives can actually be perfectly aligned. By framing the advertiser's problem as a multi-armed bandit (MAB) problem, we propose a modified Thompson Sampling (TS) algorithm that concurrently learns the optimal bidding policy and estimates the expected effect of displaying the ad while minimizing economic losses from potential sub-optimal bidding. Simulations show that not only the proposed method successfully accomplishes the advertiser's goals, but also does so at a much lower cost than more conventional experimentation policies aimed at performing causal inference.

Motivation & Objective

• To resolve the tension between learning optimal bidding policies and estimating ad exposure effects in real-time bidding (RTB) auctions.
• To develop a unified framework that simultaneously achieves both objectives without sacrificing performance.
• To minimize economic losses from sub-optimal bidding during the learning process.
• To exploit the economic structure of RTB to align exploration for learning with estimation for causal inference.
• To outperform conventional experimentation policies that prioritize causal inference at high economic cost.

Proposed method

• The advertiser's bid optimization problem is framed as a multi-armed bandit (MAB) problem.
• A modified Thompson Sampling algorithm is proposed to balance exploration and exploitation in the context of RTB.
• The method jointly learns the optimal bidding policy and estimates the expected effect of ad exposure.
• The algorithm leverages the economic structure of the auction to ensure that exploration does not incur excessive losses.
• It uses probabilistic sampling of bid values based on posterior distributions, informed by observed auction outcomes.
• The approach minimizes regret and economic loss while maintaining statistical accuracy in effect estimation.

Experimental results

Research questions

• RQ1 Can learning the optimal bidding policy and estimating the causal effect of ad exposure be simultaneously achieved in RTB?
• RQ2 How can exploration for learning be designed to minimize economic loss while still enabling accurate causal inference?
• RQ3 To what extent does the proposed method reduce economic costs compared to conventional experimentation policies?
• RQ4 Does aligning learning and estimation through the auction's economic structure improve overall advertiser performance?
• RQ5 Can a modified Thompson Sampling algorithm effectively balance exploration and exploitation in a real-time bidding environment?

Key findings

• The proposed method successfully learns optimal bidding policies and estimates ad exposure effects in parallel.
• It achieves both objectives at a significantly lower economic cost than conventional experimentation policies.
• Simulations demonstrate that the method minimizes regret and maintains high accuracy in effect estimation.
• The alignment of learning and estimation is enabled by the inherent economic structure of the RTB auction mechanism.
• The modified Thompson Sampling algorithm outperforms standard approaches in balancing exploration and exploitation under budget constraints.
• The method reduces the trade-off between learning and causal inference, making it more efficient for real-world deployment.