[论文解读] Integrating Multi-Armed Bandit, Active Learning, and Distributed Computing for Scalable Optimization

Foo Hui-Mean, Yuan‐chin Ivan Chang|arXiv (Cornell University)|Jan 2, 2026

Advanced Bandit Algorithms Research被引用 0

一句话总结

ALMAB-DC 是一个模块化框架，将主动学习、MAB 和分布式计算统一起来，以实现可扩展、具有不确定性感知的黑盒优化并具备 GPU 加速能力。

ABSTRACT

Modern optimization problems in scientific and engineering domains often rely on expensive black-box evaluations, such as those arising in physical simulations or deep learning pipelines, where gradient information is unavailable or unreliable. In these settings, conventional optimization methods quickly become impractical due to prohibitive computational costs and poor scalability. We propose ALMAB-DC, a unified and modular framework for scalable black-box optimization that integrates active learning, multi-armed bandits, and distributed computing, with optional GPU acceleration. The framework leverages surrogate modeling and information-theoretic acquisition functions to guide informative sample selection, while bandit-based controllers dynamically allocate computational resources across candidate evaluations in a statistically principled manner. These decisions are executed asynchronously within a distributed multi-agent system, enabling high-throughput parallel evaluation. We establish theoretical regret bounds for both UCB-based and Thompson-sampling-based variants and develop a scalability analysis grounded in Amdahl's and Gustafson's laws. Empirical results across synthetic benchmarks, reinforcement learning tasks, and scientific simulation problems demonstrate that ALMAB-DC consistently outperforms state-of-the-art black-box optimizers. By design, ALMAB-DC is modular, uncertainty-aware, and extensible, making it particularly well suited for high-dimensional, resource-intensive optimization challenges.

研究动机与目标

• 解决高维设置中缺乏可靠梯度时昂贵的黑盒评估挑战。
• 开发一个统一、模块化的框架，将主动学习、MAB 与分布式计算结合，实现可扩展优化。
• 利用代理建模与信息论获取函数在预算约束下引导有信息量的采样。
• 在分布式代理之间实现异步、GPU 加速评估以提升吞吐量。
• 提供理论上的遗憾界与可扩展性分析，支持分布式、具有不确定性感知的优化。

提出的方法

• 将优化视为在不确定性下由贝叶斯代理驱动的序列决策过程。
• 使用获取函数（如熵、期望改进、互信息）来选择下一个有信息量的输入。
• 将基于赌博机的控制器（UCB、Thompson 采样）用于在候选评估之间分配计算资源。
• 在计算节点之间异步分发评估，代理建模和评估可选地使用 GPU 加速。
• 迭代更新代理模型和赌博机统计量，以精炼未来查询。
• 为分布式、异步设置提供理论遗憾界与可扩展性分析。

Figure 1: ALMAB-DC Framework: Integration of Active Learning, Multi-Armed Bandits, and Distributed Computing through Bayesian Surrogate Modeling

实验结果

研究问题

• RQ1 如何将主动学习与赌博机策略整合，以在分布式环境中实现可扩展、信息高效的黑盒优化？
• RQ2 在异步反馈与通信开销下，ALMAB-DC 的理论遗憾与可扩展性性质是什么？
• RQ3 GPU 加速的分布式评估如何影响高成本优化任务的吞吐量与收敛性？
• RQ4 在可扩展性能方面，代理数量与协调成本之间的最优平衡是什么？
• RQ5 ALMAB-DC 是否能够在保持不确定性量化的前提下，适应多保真度和异构计算环境？

主要发现

• ALMAB-DC 通过在一个模块化管线中统一 AL、MAB 与 DC，提升可扩展性与样本效率。
• 该框架在分布式、异步设置中提供基于 UCB 与 Thompson Sampling 的变体的遗憾界。
• 基于阿姆达尔定律与古斯塔夫森定律的可扩展性分析为资源分配提供指引，并预测加速比与局限性。
• 利用 GPU 加速进行代理建模、后验更新和获取优化以提升吞吐量。
• 在综合的合成基准、强化学习任务与科学仿真中的实验结果显示，与最先进的黑盒优化器相比具有有竞争力的性能。

Figure 2: ALMAB-DC Architecture Pipeline: The framework integrates Active Learning (AL), Multi-Armed Bandits (MAB), and Distributed Computing (DC) into a modular pipeline. Decision modules (top) include the Unlabeled Data Pool, Active Learner, and Bandit Controller, which guide candidate selection a