[论文解读] Adaptive Combinatorial Experimental Design: Pareto Optimality for Decision-Making and Inference

Hongrui Xie, Junyu Cao+1 more|arXiv (Cornell University)|Feb 27, 2026

Advanced Bandit Algorithms Research被引用 0

一句话总结

本文提出Pareto最优学习用于自适应组合枪信道博弈的情境，提出MixCombKL（全盲带）和MixCombUCB（半盲带），并在两种反馈 regimes 下给出有限时 regret 和 gap 估计保证。

ABSTRACT

In this paper, we provide the first investigation into adaptive combinatorial experimental design, focusing on the trade-off between regret minimization and statistical power in combinatorial multi-armed bandits (CMAB). While minimizing regret requires repeated exploitation of high-reward arms, accurate inference on reward gaps requires sufficient exploration of suboptimal actions. We formalize this trade-off through the concept of Pareto optimality and establish equivalent conditions for Pareto-efficient learning in CMAB. We consider two relevant cases under different information structures, i.e., full-bandit feedback and semi-bandit feedback, and propose two algorithms MixCombKL and MixCombUCB respectively for these two cases. We provide theoretical guarantees showing that both algorithms are Pareto optimal, achieving finite-time guarantees on both regret and estimation error of arm gaps. Our results further reveal that richer feedback significantly tightens the attainable Pareto frontier, with the primary gains arising from improved estimation accuracy under our proposed methods. Taken together, these findings establish a principled framework for adaptive combinatorial experimentation in multi-objective decision-making.

研究动机与目标

• Motivate the study of regret versus inference trade-offs in combinatorial bandits (CMAB).
• Formalize Pareto optimality as a framework for balancing regret and reward-gap estimation.
• Develop Pareto-optimal algorithms for two feedback models (full-bandit and semi-bandit).
• Provide finite-time guarantees for both regret and estimation errors under each feedback regime.

提出的方法

• Model CMAB with base arms and super arms under full-bandit and semi-bandit feedback.
• Define Pareto optimality and Pareto frontier to capture trade-offs between regret and estimation error.
• Develop MixCombKL for full-bandit feedback using KL-divergence guided online stochastic mirror descent on a simplex embedding.
• Develop MixCombUCB for semi-bandit feedback using a UCB-based approach with an initialization phase and an optimization oracle.
• Provide finite-sample bounds for estimation errors of super-arm gaps and base-arm gaps, and regret bounds for both algorithms.
• Establish necessary and sufficient conditions for Pareto optimality and relate richness of feedback to frontier tightness.

实验结果

研究问题

• RQ1 What is the trade-off between regret minimization and statistical power for estimating reward gaps in CMAB?
• RQ2 Can Pareto optimal policies be characterized and achieved in CMAB under different feedback regimes?
• RQ3 How do full-bandit and semi-bandit feedback impact the Pareto frontier and learning guarantees?
• RQ4 What are the finite-time estimation and regret guarantees for MixCombKL and MixCombUCB?
• RQ5 What are the necessary and sufficient conditions for Pareto optimality in CMAB settings?

主要发现

• MixCombKL achieves Pareto-optimal trade-offs under full-bandit feedback with finite-time gap estimation guarantees and regret bounds.
• MixCombUCB achieves Pareto-optimal trade-offs under semi-bandit feedback with finite-time gap estimation guarantees and regret bounds.
• Semi-bandit feedback yields a sharper Pareto frontier than full-bandit feedback due to improved estimation accuracy, while regret scales similarly under the proposed algorithms.
• The paper provides explicit finite-sample bounds for both super-arm gap estimation and base-arm gap estimation, along with regret bounds.
• Pareto optimality is characterized by conditions linking estimation error and regret, applicable to both feedback models.