[Paper Review] Multi-Armed Bandit Dynamic Beam Zooming for mmWave Alignment and Tracking

Nathan Blinn, Matthieu R. Bloch|arXiv (Cornell University)|Sep 7, 2022

Millimeter-Wave Propagation and Modeling4 citations

TL;DR

This paper proposes Successive Subtree Elimination (SSE), a low-complexity multi-armed bandit algorithm for mmWave beam alignment using hierarchical codebooks. By exploiting unimodal reward structure and fixed-confidence best-arm identification, SSE achieves near-optimal performance with minimal channel state information, significantly outperforming other low-complexity methods in low SNR regimes while enabling offline parameter tuning via closed-form sample complexity analysis.

ABSTRACT

We propose an Integrated Sensing and Communication (ISAC) algorithm that exploits the structure of a hierarchical codebook of beamforming vectors using a best-arm identification Multi-Armed Bandit (MAB) approach for initial alignment and tracking of a Mobile Entity (ME). The algorithm, called Dynamic Beam Zooming (DBZ), performs beam adjustments that mitigate the severe outages associated with wireless mmWave systems and allow for adaptive control of the parameters governing communications. We analyze the sample complexity of DBZ and use it to inform how the algorithm adapts to the nonstationary MAB statistics based on ME motion and Signal-to-Noise Ratio (SNR). We perform extensive simulations to validate the approach and demonstrate that DBZ is competitive against existing Bayesian algorithms, without requiring channel multipath or fading knowledge. In particular, DBZ outperforms other low-complexity algorithms in the low SNR regime. We also illustrate the efficacy of DBZ in standardized rural and urban scenarios using NYU Sim.

Motivation & Objective

• To address the high training overhead of exhaustive beam sweeping in mmWave systems by enabling faster initial beam alignment.
• To develop a low-complexity alternative to Bayesian and deep learning-based beam alignment methods that require extensive channel state information or training.
• To leverage the hierarchical structure of beam codebooks and the approximate unimodal nature of beam gain to accelerate convergence in beam selection.
• To provide theoretical guarantees on sample complexity and correctness for a fixed-confidence best-arm identification setting in mmWave beam alignment.
• To enable offline tuning of algorithm parameters through a closed-form sample complexity expression.

Proposed method

• Formulates mmWave beam alignment as a fixed-confidence best-arm identification problem in a multi-armed bandit framework, treating codebook beams as arms.
• Introduces Successive Subtree Elimination (SSE), a sequential sampling strategy that eliminates subtrees of beams with low confidence in being optimal.
• Uses confidence intervals based on Hoeffding's inequality to bound the true mean reward of each beam, with bounds updated after each channel observation.
• Employs a hierarchical codebook structure to exploit spatial correlation and reduce search space, enabling efficient beam steering toward the strongest path.
• Applies a unimodal reward assumption to justify pruning beams that cannot be optimal, improving convergence speed.
• Derives a closed-form sample complexity expression for SSE, enabling offline optimization of design parameters like confidence level and error tolerance.

Experimental results

Research questions

• RQ1 Can a low-complexity multi-armed bandit algorithm achieve near-optimal beam alignment performance without requiring full channel state information?
• RQ2 How does the hierarchical codebook structure interact with beam reward unimodality to improve convergence speed in mmWave beam alignment?
• RQ3 What is the theoretical sample complexity of a fixed-confidence best-arm identification algorithm in the context of mmWave beam alignment?
• RQ4 How does SSE compare to state-of-the-art Bayesian and deep learning-based beam alignment methods in low SNR conditions?
• RQ5 Can the sample complexity of the algorithm be expressed in closed form to allow offline parameter tuning?

Key findings

• SSE achieves performance comparable to state-of-the-art Bayesian algorithms like HPM but with significantly lower computational complexity and without requiring channel state information.
• In low SNR regimes, SSE outperforms other low-complexity algorithms such as UBA and HBA, demonstrating superior robustness to noise.
• The algorithm's sample complexity is derived in closed form, enabling offline tuning of parameters such as confidence level and error tolerance.
• Theoretical analysis proves that SSE correctly identifies the optimal beam with high probability under the fixed-confidence setting.
• The unimodal structure of beam gains allows effective pruning of suboptimal beam subtrees, accelerating convergence.
• Extensive simulations over slow-fading channels confirm the theoretical findings and validate the algorithm’s robustness and efficiency.