[Paper Review] Computer Vision Aided mmWave Beam Alignment in V2X Communications

Weihua Xu, Feifei Gao+3 more|arXiv (Cornell University)|Jul 23, 2022

Millimeter-Wave Propagation and Modeling4 citations

TL;DR

This paper proposes a computer vision-aided mmWave beam alignment framework for V2X communications that leverages onboard camera images from mobile users to enable pilot-free beam pair selection and dynamic beam coherence time (BCT) prediction. By using 3D object detection and a deep neural network (DNN), the method achieves high alignment accuracy with lower hardware cost and communication overhead than LIDAR- or BS-based approaches.

ABSTRACT

Visual information, captured for example by cameras, can effectively reflect the sizes and locations of the environmental scattering objects, and thereby can be used to infer communications parameters like propagation directions, receiver powers, as well as the blockage status. In this paper, we propose a novel beam alignment framework that leverages images taken by cameras installed at the mobile user. Specifically, we utilize 3D object detection techniques to extract the size and location information of the dynamic vehicles around the mobile user, and design a deep neural network (DNN) to infer the optimal beam pair for transceivers without any pilot signal overhead. Moreover, to avoid performing beam alignment too frequently or too slowly, a beam coherence time (BCT) prediction method is developed based on the vision information. This can effectively improve the transmission rate compared with the beam alignment approach with the fixed BCT. Simulation results show that the proposed vision based beam alignment methods outperform the existing LIDAR and vision based solutions, and demand for much lower hardware cost and communication overhead.

Motivation & Objective

• To address the high hardware and communication overhead of traditional beam alignment in mmWave V2X systems.
• To eliminate reliance on LIDAR or BS-based visual perception, which incur high cost and privacy concerns.
• To enable accurate, low-overhead beam alignment using only onboard camera data from mobile users.
• To predict beam coherence time dynamically based on visual scene changes, improving transmission efficiency.
• To achieve robust beam alignment performance in both LOS and NLOS scenarios with minimal pilot signaling.

Proposed method

• Utilizes 3D object detection on camera images captured by mobile users to extract size and location of surrounding vehicles.
• Employs a deep neural network (DNN) to infer optimal beam pairs directly from visual features, eliminating pilot signal overhead.
• Introduces a vision-based beam coherence time prediction (VPBCT) method using sequential images to adapt alignment frequency to environmental dynamics.
• Designs a vision-based beam alignment (VBALA) framework that leverages MS-centric perception to avoid MS identification and communication overhead.
• Uses scene image sequences to train a Siamese-based perception network (SIBPN) for accurate BCT prediction.
• Applies visual features (x, y, z coordinates of detected objects) as input to a vision learning framework (VLF) independent of MS location error.

Experimental results

Research questions

• RQ1 Can onboard camera images from mobile users replace LIDAR or BS-based perception for mmWave beam alignment?
• RQ2 How can visual perception from mobile users enable pilot-free beam pair selection with high accuracy?
• RQ3 Can beam coherence time be predicted dynamically using visual input to improve transmission efficiency?
• RQ4 How does MS-centric vision perception compare to BS-centric or LIDARD-based methods in terms of accuracy and overhead?
• RQ5 What is the impact of MS location error on beam alignment performance when using vision-based methods?

Key findings

• The proposed VBALA method achieves approximately 4% higher average ATRR than BMBA in NLOS scenarios due to reduced communication overhead and improved robustness.
• VPBCT improves transmission rate by 6.0% (73.2% vs. 67.2%) compared to fixed BCT with M_f=2 and T_b=1/3 T_d.
• SABA, a variant using only 3D object coordinates, shows strong robustness to MS location error (σ_c up to 0.5m) due to location-independent visual features.
• When MS location error exceeds 0.16m, VBALU outperforms VBALA, BMBA, and LBA in both LOS and NLOS scenarios.
• The BCT prediction accuracy (BCTPA) of VPBCT reaches about 60% after convergence, demonstrating effective adaptation to environmental dynamics.
• VBALA and VBALU achieve better beam alignment performance than LBA and BMBA with no additional communication cost, especially in NLOS conditions.