[Paper Review] Vehicle Communication Strategies for Simulated Highway Driving

Cinjon Resnick, Ilya Kulikov+2 more|arXiv (Cornell University)|Apr 19, 2018

Reinforcement Learning in Robotics7 references3 citations

TL;DR

This paper investigates vehicle-to-vehicle (V2V) communication in simulated highway driving using multi-agent reinforcement learning. It proposes an engineered V2V protocol and compares it to no communication and emergent communication approaches, finding that V2V communication significantly improves safety and efficiency—especially in adverse (foggy) conditions—boosting per-car success rates to 91% and 85.7% in sunny and foggy evaluations, respectively.

ABSTRACT

Interest in emergent communication has recently surged in Machine Learning. The focus of this interest has largely been either on investigating the properties of the learned protocol or on utilizing emergent communication to better solve problems that already have a viable solution. Here, we consider self-driving cars coordinating with each other and focus on how communication influences the agents' collective behavior. Our main result is that communication helps (most) with adverse conditions.

Motivation & Objective

• To investigate how communication influences collective behavior in autonomous vehicle coordination under adverse driving conditions.
• To evaluate the effectiveness of an engineered V2V communication protocol versus no communication and emergent communication in a simulated highway environment.
• To explore whether learned communication protocols can outperform or complement fixed engineering standards in safety-critical driving scenarios.
• To assess the robustness of trained policies when evaluated under different environmental conditions, particularly visibility degradation.
• To compare the performance of fixed V2V protocols with end-to-end learned communication mechanisms in a real-world-relevant simulation.

Proposed method

• A custom Box2D-based highway simulation environment was created with 12 randomly initialized vehicles, each with a unique exit goal and dynamic starting positions.
• A shared deep reinforcement learning policy was trained using a reward function that rewarded reaching the exit (+60), penalized crashes or overshooting (-60), and applied a -0.5 penalty per time step.
• The observation space included 40-dimensional state vectors (e.g., position, velocity, lidar data) and was augmented with V2V messages when communication was enabled.
• The V2V protocol transmitted 21 structured message fields, including speed, heading, acceleration, and position, based on the proposed V2V standard.
• Two emergent communication variants were explored: 'continuous' (learned real-valued message vectors) and 'select' (learned selection of V2V message components).
• Training was conducted under both sunny and foggy conditions, with evaluation metrics including success rate, episode length, and failure rates across environments.

Experimental results

Research questions

• RQ1 Does engineered V2V communication improve safety and efficiency in autonomous vehicle coordination under adverse driving conditions?
• RQ2 How does the performance of a fixed V2V protocol compare to a no-communication baseline and end-to-end learned communication in a simulated highway setting?
• RQ3 Can emergent communication protocols outperform or stabilize the performance of engineered V2V protocols in complex driving scenarios?
• RQ4 How does training in adverse conditions affect the robustness and generalization of communication-enhanced policies?
• RQ5 What components of the V2V protocol are most critical for performance, as revealed by interpretable emergent communication?

Key findings

• In sunny conditions, the V2V protocol achieved a per-car success rate of 91.03%, compared to 79.51% for the no-communication baseline.
• In foggy conditions, the V2V protocol maintained a per-car success rate of 85.71%, significantly outperforming the baseline’s 70.31%.
• The V2V protocol improved episode success rate to 52.57% in sunny conditions and 46.06% in foggy conditions, compared to 23.40% and 25.68% for the baseline.
• The V2V model reduced average episode failure length by 7.1% in sunny conditions and 4.4% in foggy conditions, indicating more efficient navigation.
• Models trained in sunny conditions generalized better to foggy evaluations than those trained in foggy conditions, due to the 90% sunny evaluation bias.
• Emergent communication approaches ('continuous' and 'select') failed to converge to effective policies and often degenerated into non-productive behaviors such as circular motion.