[Paper Review] Augmented Reality-Based Advanced Driver-Assistance System for Connected Vehicles

Ziran Wang, Kyungtae Han+1 more|arXiv (Cornell University)|Aug 31, 2020

Traffic control and management25 references4 citations

TL;DR

This paper proposes an augmented reality (AR)-based advanced driver-assistance system (ADAS) for connected vehicles that visualizes cooperative guidance at unsignalized intersections using slot reservation and feedforward/feedback control. The system enables human-driven vehicles to cross intersections without stopping, reducing travel time by 20% and fuel consumption by 23.7% in human-in-the-loop simulations using Unity and real driver inputs.

ABSTRACT

With the development of advanced communication technology, connected vehicles become increasingly popular in our transportation systems, which can conduct cooperative maneuvers with each other as well as road entities through vehicle-to-everything communication. A lot of research interests have been drawn to other building blocks of a connected vehicle system, such as communication, planning, and control. However, less research studies were focused on the human-machine cooperation and interface, namely how to visualize the guidance information to the driver as an advanced driver-assistance system (ADAS). In this study, we propose an augmented reality (AR)-based ADAS, which visualizes the guidance information calculated cooperatively by multiple connected vehicles. An unsignalized intersection scenario is adopted as the use case of this system, where the driver can drive the connected vehicle crossing the intersection under the AR guidance, without any full stop at the intersection. A simulation environment is built in Unity game engine based on the road network of San Francisco, and human-in-the-loop (HITL) simulation is conducted to validate the effectiveness of our proposed system regarding travel time and energy consumption.

Motivation & Objective

• To address the lack of human-centered design in ADAS interfaces for connected vehicles, particularly in visualizing cooperative guidance.
• To enable safe, efficient, and stop-free crossing of unsignalized intersections through AR visualization of cooperative vehicle maneuvers.
• To develop a human-machine interface that integrates planning and control modules with AR HMI to improve driver comprehension and system usability.
• To validate the system’s effectiveness in reducing travel time and energy consumption through human-in-the-loop simulation.

Proposed method

• The system uses a digital twin architecture with V2X communication to enable real-time coordination among connected vehicles (CAVs and human-driven).
• A slot reservation algorithm assigns time slots to vehicles approaching an unsignalized intersection to prevent conflicts and ensure smooth passage.
• A feedforward/feedback control algorithm maintains safe inter-vehicle distances by adjusting speed based on reference vehicles and reserved slots.
• The AR HMI overlays guidance cues (e.g., speed commands and trajectory indicators) directly onto the driver’s windshield view via a heads-up display.
• Unity game engine was used to simulate a San Francisco road network with human participants in the loop to evaluate system performance.
• The system was evaluated using travel time and energy consumption metrics, with fuel use estimated via the MOVESTAR model.

Experimental results

Research questions

• RQ1 How can AR-based HMI effectively visualize cooperative vehicle guidance to improve driver understanding and performance at unsignalized intersections?
• RQ2 Can a slot reservation mechanism enable stop-free, coordinated crossing of multiple vehicles at unsignalized intersections through V2X communication?
• RQ3 To what extent does AR-guided cooperative driving reduce travel time and energy consumption compared to traditional signalized intersections?
• RQ4 How does the integration of feedforward/feedback control with AR HMI improve safety and efficiency in mixed-traffic environments?

Key findings

• The proposed AR-based ADAS reduced average travel time by 20% compared to traditional signalized intersections in human-in-the-loop simulations.
• The system achieved an average reduction of 23.7% in fuel consumption, as estimated using the open-source MOVESTAR model for gasoline-powered vehicles.
• The feedforward/feedback control algorithm successfully maintained safe inter-vehicle distances, with NPC vehicles decelerating appropriately when following the ego vehicle.
• Slot reservation was dynamically managed per intersection, with vehicles receiving new slots upon entering the next link, enabling continuous coordination.
• The AR HMI enabled human drivers to cross intersections without stopping, maintaining stable speeds and improving overall traffic efficiency.
• The system demonstrated that human-centered design of ADAS—where planning and control are tailored to HMI—can significantly enhance usability and performance.