[Paper Review] Multinomial Logit Bandit with Low Switching Cost

Kefan Dong, Yingkai Li+2 more|arXiv (Cornell University)|Jul 9, 2020

Advanced Bandit Algorithms Research4 citations

TL;DR

This paper proposes AT-DUCB and FH-DUCB algorithms for the multinomial logit bandit problem with low switching cost, minimizing assortment switches while achieving near-optimal regret. It establishes tight bounds: AT-DUCB incurs $O(N\log T)$ assortment switches, nearly matching the $\Omega(N\log T / \log\log T)$ lower bound, and FH-DUCB achieves $O(N\log\log T)$ switches in fixed-horizon settings, matching the asymptotic lower bound.

ABSTRACT

We study multinomial logit bandit with limited adaptivity, where the algorithms change their exploration actions as infrequently as possible when achieving almost optimal minimax regret. We propose two measures of adaptivity: the assortment switching cost and the more fine-grained item switching cost. We present an anytime algorithm (AT-DUCB) with $O(N \log T)$ assortment switches, almost matching the lower bound $Ω(\frac{N \log T}{ \log \log T})$. In the fixed-horizon setting, our algorithm FH-DUCB incurs $O(N \log \log T)$ assortment switches, matching the asymptotic lower bound. We also present the ESUCB algorithm with item switching cost $O(N \log^2 T)$.

Motivation & Objective

• To minimize assortment and item switching costs in multinomial logit bandit problems while maintaining near-optimal regret.
• To address the practical constraint that frequent assortment changes are costly in retail and online advertising.
• To develop anytime and fixed-horizon algorithms with provably low switching costs.
• To establish theoretical lower bounds on assortment switching cost for any algorithm achieving near-optimal regret.
• To analyze the trade-off between regret and switching cost using two measures: assortment switching and item switching cost.

Proposed method

• Proposes AT-DUCB, an anytime algorithm that uses UCB-based exploration and deferred updates to reduce assortment switches to $O(N\log T)$.
• Introduces FH-DUCB for fixed-horizon settings, achieving $O(N\log\log T)$ assortment switches by leveraging time-horizon knowledge.
• Designs ESUCB with $O(N\log^2 T)$ item switching cost, later modified to achieve $O(N\log T)$ item switches via non-reinitialization of counters.
• Uses a deferred update mechanism in UCB computation to limit the number of value updates and thus reduce switching events.
• Applies concentration inequalities and high-probability event analysis to bound regret and switching costs under uncertainty.
• Employs a symmetric difference operator $|S_t \oplus S_{t+1}|$ to define item switching cost, enabling fine-grained control over changes.

Experimental results

Research questions

• RQ1 What is the minimal number of assortment switches required by any algorithm achieving near-optimal regret in the MNL bandit setting?
• RQ2 Can an anytime algorithm achieve both low regret and low assortment switching cost without knowing the time horizon in advance?
• RQ3 How does the item switching cost compare to assortment switching cost in terms of theoretical bounds and algorithmic design?
• RQ4 Can the fixed-horizon setting allow for tighter switching cost bounds than the anytime setting?
• RQ5 What modifications to existing UCB-style algorithms reduce item switching cost while preserving regret guarantees?

Key findings

• AT-DUCB achieves $O(N\log T)$ assortment switches while maintaining near-optimal regret, nearly matching the lower bound of $\Omega(N\log T / \log\log T)$.
• FH-DUCB achieves $O(N\log\log T)$ assortment switches in the fixed-horizon setting, matching the asymptotic lower bound.
• The modified ESUCB algorithm reduces item switching cost from $O(N\log^2 T)$ to $O(N\log T)$ by avoiding reinitialization of counters in the Check subroutine.
• A lower bound of $\Omega(N\log T / \log\log T)$ on assortment switching cost is proven for any anytime algorithm with near-optimal regret.
• The item switching cost is always at least as large as the assortment switching cost, and at most $\min\{2K,N\}$ times larger, establishing a tight relationship between the two measures.