[论文解读] Bandit Inspired Beam Searching Scheme for mmWave High-Speed Train Communications

Jun-Bo Wang, Ming Cheng+5 more|arXiv (Cornell University)|Oct 14, 2018

Millimeter-Wave Propagation and Modeling被引用 9

一句话总结

本文提出了一种受多臂赌博机（MAB）启发的波束搜索方案，用于毫米波高速列车（HST）通信，以最小化信道估计算 overhead。通过利用历史传播知识和强化学习，该方案在仅两次HST通行的情况下，显著减少了路径测量次数，同时实现了接近理论极限的频谱效率，且对训练数据量要求极低，显著优于传统方法。

ABSTRACT

High-speed trains (HSTs) are being widely deployed around the world. To meet the high-rate data transmission requirements on HSTs, millimeter wave (mmWave) HST communications have drawn increasingly attentions. To realize sufficient link margin, mmWave HST systems employ directional beamforming with large antenna arrays, which results in that the channel estimation is rather time-consuming. In HST scenarios, channel conditions vary quickly and channel estimations should be performed frequently. Since the period of each transmission time interval (TTI) is too short to allocate enough time for accurate channel estimation, the key challenge is how to design an efficient beam searching scheme to leave more time for data transmission. Motivated by the successful applications of machine learning, this paper tries to exploit the similarities between current and historical wireless propagation environments. Using the knowledge of reinforcement learning, the beam searching problem of mmWave HST communications is formulated as a multi-armed bandit (MAB) problem and a bandit inspired beam searching scheme is proposed to reduce the number of measurements as many as possible. Unlike the popular deep learning methods, the proposed scheme does not need to collect and store a massive amount of training data in advance, which can save a huge amount of resources such as storage space, computing time, and power energy. Moreover, the performance of the proposed scheme is analyzed in terms of regret. The regret analysis indicates that the proposed schemes can approach the theoretical limit very quickly, which is further verified by simulation results.

研究动机与目标

• 为解决毫米波HST系统中因信道快速变化而导致的频繁、时间受限的信道估计算问题。
• 在不依赖大规模训练数据集的前提下，减少准确信道估计算所需的波束测量次数。
• 通过最小化短TTI时隙内波束搜索所花费的时间，实现高效的数据传输。
• 利用历史无线传播模式，加速对最优波束配置的学习。
• 设计一种低复杂度、高数据效率的波束搜索算法，适用于高频移动的毫米波环境。

提出的方法

• 将波束搜索问题建模为多臂赌博机（MAB）问题，其中每个波束方向视为一个“臂”，奖励对应于频谱效率。
• 采用受赌博机启发的学习算法，根据累积奖励动态选择波束，实现探索与利用之间的平衡。
• 通过奖励机制收集并更新HST通行过程中可用传播路径的知识。
• 进行遗憾分析，理论上界定了性能损失，表明其随时间呈对数增长，且与波束数量呈线性依赖关系。
• 通过避免使用深度学习，无需预先收集的训练数据，从而降低了存储、计算和能耗成本。
• 采用虚拟信道矩阵（VCM）模型来表示波束特定的信道增益，并优化波束选择。

实验结果

研究问题

• RQ1 基于强化学习的波束搜索方案能否减少毫米波HST信道估计算所需的测量次数？
• RQ2 在高频移动的HST环境中，受赌博机启发的算法能多快学习到最优波束配置？
• RQ3 每次时隙的测量次数与收敛至最优频谱效率速度之间的权衡关系如何？
• RQ4 在HST场景中，历史传播知识能在多大程度上减少重复信道估计算的需求？
• RQ5 与传统波束搜索方法相比，所提出的方案在频谱效率和学习效率方面表现如何？

主要发现

• 所提出的方案仅在两次HST通行内即达到接近理论频谱效率极限，表明其能快速学习传播环境。
• 当HST第二次通行时，遗憾值增长极为缓慢，表明几乎所有最优波束均已识别并被有效利用。
• 当每次时隙的测量数M=4时，方案仍与理论极限存在性能差距，表明M=4不足以实现完全探索。
• 增加每次时隙的测量次数可加速收敛，从而在首次通行期间缩小与理论极限的差距。
• 算法的遗憾值随时间呈对数增长，且与波束数量呈线性关系，验证了理论性能边界。
• 该方案显著优于改进的顺序波束搜索算法，尤其是在首次通行后，得益于对历史数据的有效利用。