[Paper Review] Augmented Informative Cooperative Perception.

Pengyuan Zhou, Pranvera Kortoçi+8 more|arXiv (Cornell University)|Jan 14, 2021

Vehicular Ad Hoc Networks (VANETs)40 references4 citations

TL;DR

This paper proposes Augmented Informative Cooperative Perception (AICP), a low-latency, high-efficiency system that optimizes data sharing among connected vehicles by filtering and prioritizing only the most informative visual data for augmented reality head-up displays. By combining a lightweight routing protocol and a fitness-based sorting algorithm, AICP reduces network load and achieves only 12.6 ms additional latency while decreasing channel busy time through intelligent data filtering.

ABSTRACT

Connected vehicles, whether equipped with advanced driver-assistance systems or fully autonomous, are currently constrained to visual information in their lines-of-sight. A cooperative perception system among vehicles increases their situational awareness by extending their perception ranges. Existing solutions imply significant network and computation load, as well as high flow of not-always-relevant data received by vehicles. To address such issues, and thus account for the inherently diverse informativeness of the data, we present Augmented Informative Cooperative Perception (AICP) as the first fast-filtering system which optimizes the informativeness of shared data at vehicles. AICP displays the filtered data to the drivers in augmented reality head-up display. To this end, an informativeness maximization problem is presented for vehicles to select a subset of data to display to their drivers. Specifically, we propose (i) a dedicated system design with custom data structure and light-weight routing protocol for convenient data encapsulation, fast interpretation and transmission, and (ii) a comprehensive problem formulation and efficient fitness-based sorting algorithm to select the most valuable data to display at the application layer. We implement a proof-of-concept prototype of AICP with a bandwidth-hungry, latency-constrained real-life augmented reality application. The prototype realizes the informative-optimized cooperative perception with only 12.6 milliseconds additional latency. Next, we test the networking performance of AICP at scale and show that AICP effectively filter out less relevant packets and decreases the channel busy time.

Motivation & Objective

• To address the high network and computation overhead caused by transmitting irrelevant or redundant data in cooperative perception systems among connected vehicles.
• To improve situational awareness by selectively sharing only the most informative visual data from connected vehicles.
• To reduce latency and bandwidth usage in cooperative perception by filtering out less relevant data before transmission.
• To integrate the filtered, high-informativeness data into augmented reality head-up displays for real-time driver feedback.
• To design a system that balances computational efficiency, low-latency transmission, and perceptual relevance in dynamic vehicular environments.

Proposed method

• Designing a custom data structure and lightweight routing protocol to enable fast data encapsulation, interpretation, and transmission across vehicles.
• Formulating an informativeness maximization problem to select the most valuable subset of visual data for display.
• Implementing a fitness-based sorting algorithm at the application layer to rank and prioritize data based on its perceived informativeness.
• Integrating the selected data into an augmented reality head-up display for real-time visualization to drivers.
• Optimizing data transmission by filtering out low-informativeness packets before network dissemination.
• Evaluating system performance under real-life, bandwidth-hungry, latency-constrained AR conditions to validate efficiency and responsiveness.

Experimental results

Research questions

• RQ1 How can cooperative perception systems efficiently filter out irrelevant visual data to reduce network and computational load in connected vehicles?
• RQ2 What criteria and methods can be used to objectively measure and maximize the informativeness of shared visual data in vehicular networks?
• RQ3 To what extent can a lightweight, fitness-based data selection mechanism reduce latency and channel utilization in AR-enabled cooperative perception systems?
• RQ4 How does the integration of filtered, high-informativeness data into augmented reality head-up displays affect real-time driver perception and system responsiveness?
• RQ5 Can a scalable cooperative perception system maintain low latency and high data relevance under real-world network constraints?

Key findings

• The AICP prototype achieves only 12.6 milliseconds of additional latency when integrating informative data into augmented reality head-up displays.
• AICP significantly reduces channel busy time by filtering out less relevant data packets, improving network efficiency.
• The system’s lightweight routing protocol and custom data structure enable fast data interpretation and transmission with minimal overhead.
• The fitness-based sorting algorithm effectively prioritizes data with high perceptual relevance, enhancing driver situational awareness.
• The system demonstrates scalability in large-scale network testing, maintaining low latency and high data relevance.
• The integration of AICP with AR HUDs proves feasible and efficient under real-life, bandwidth-constrained conditions.