[Paper Review] Constrained Thompson Sampling for Wireless Link Optimization

Vidit Saxena, Joseph E. Gonzalez+3 more|arXiv (Cornell University)|Feb 28, 2019

Advanced Bandit Algorithms Research26 references4 citations

TL;DR

This paper proposes Con-TS, a constrained Thompson Sampling algorithm for wireless link rate selection that balances throughput maximization with probabilistic latency constraints. By modeling latency as a high-probability linear constraint and using Bayesian posterior sampling with LP relaxation, Con-TS achieves $O(\sqrt{KT})$ expected constraint violations and $O(\sqrt{KT\log K})$ regret, outperforming state-of-the-art bandit methods in simulations across diverse wireless environments.

ABSTRACT

Wireless communication systems operate in complex time-varying environments. Therefore, selecting the optimal configuration parameters in these systems is a challenging problem. For wireless links, \emph{rate selection} is used to select the optimal data transmission rate that maximizes the link throughput subject to an application-defined latency constraint. We model rate selection as a stochastic multi-armed bandit (MAB) problem, where a finite set of transmission rates are modeled as independent bandit arms. For this setup, we propose Con-TS, a novel constrained version of the Thompson sampling algorithm, where the latency requirement is modeled by a high-probability linear constraint. We show that for Con-TS, the expected number of constraint violations over T transmission intervals is upper bounded by O(\sqrt{KT}), where K is the number of available rates. Further, the expected loss in cumulative throughput compared to the optimal rate selection scheme (i.e., the egret is also upper bounded by O(\sqrt{KT \log K}). Through numerical simulations, we demonstrate that Con-TS significantly outperforms state-of-the-art bandit schemes for rate selection.

Motivation & Objective

• To address the challenge of optimizing wireless link throughput while respecting application-level latency constraints.
• To model rate selection as a constrained multi-armed bandit problem with a probabilistic linear constraint on packet success rate.
• To develop a novel Thompson Sampling-based algorithm (Con-TS) that handles soft constraints efficiently and provides finite-time performance guarantees.
• To demonstrate through simulations that Con-TS outperforms existing bandit algorithms like UTS and Con-KL-UCB in throughput-violation ratio and constraint violation count.
• To show that the proposed formulation as a relaxed knapsack problem enables efficient, polynomial-time optimization and broader applicability to other wireless optimization problems.

Proposed method

• Formulate rate selection as a stochastic multi-armed bandit problem with K transmission rates as arms.
• Model the latency constraint as a high-probability linear constraint on the packet success rate, ensuring a minimum acceptable reliability level.
• Apply a linear programming relaxation to the knapsack formulation of the problem, enabling efficient computation of feasible policies.
• Propose Con-TS, a probabilistically constrained variant of Thompson Sampling that samples from posterior distributions while respecting the constraint with high probability.
• Use Bayesian updating to refine estimates of each rate's success probability and throughput over time.
• Integrate a prior-independent design that ensures finite-time regret and violation bounds without requiring prior knowledge of the true parameters.

Experimental results

Research questions

• RQ1 Can a Thompson Sampling-based algorithm be effectively adapted to handle probabilistic linear constraints in wireless rate selection?
• RQ2 What are the finite-time performance bounds for constraint violations and regret in a latency-constrained multi-armed bandit setting?
• RQ3 How does Con-TS compare to state-of-the-art bandit algorithms like UTS and Con-KL-UCB in terms of throughput and constraint compliance?
• RQ4 In what wireless propagation environments does Con-TS demonstrate superior robustness and performance?
• RQ5 Can the knapsack formulation with LP relaxation be effectively used to model and solve latency-constrained rate selection problems?

Key findings

• Con-TS achieves an upper bound of $O(\sqrt{KT})$ on the expected cumulative number of constraint violations over $T$ transmission intervals.
• The expected regret of Con-TS is bounded by $O(\sqrt{KT\log K})$, indicating strong convergence to the optimal rate policy.
• In the Gradual, Lossy, and Linear WiFi environments, Con-TS achieves a throughput-violation ratio approximately double that of the next-best algorithm, UTS.
• For the Gradual environment, Con-TS accumulates less than half the cumulative constraint violations compared to competing algorithms, demonstrating strong constraint adherence.
• In the Steep environment, where the constraint is trivially satisfied due to high success probabilities, Con-TS performs comparably to UTS and Con-KL-UCB, with all three achieving low violation counts.
• Con-TS consistently maintains low constraint violations across all scenarios, while UTS—being unconstrained—exhibits a linear violation rate in non-trivial environments.