[Paper Review] AgentsCoMerge: Large Language Model Empowered Collaborative Decision Making for Ramp Merging

Senkang Hu, Zhengru Fang+5 more|arXiv (Cornell University)|Aug 7, 2024

Collaboration in agile enterprises4 citations

TL;DR

This paper proposes AgentsCoMerge, a large language model (LLM)-powered collaborative decision-making framework for multi-lane ramp merging in connected and autonomous vehicles (CAVs). It integrates scene understanding, hierarchical planning, inter-agent communication, and reinforcement reflection guided training, achieving a 97.67% reduction in collision rate and 90.99% lower L2 trajectory error compared to baseline methods.

ABSTRACT

Ramp merging is one of the bottlenecks in traffic systems, which commonly cause traffic congestion, accidents, and severe carbon emissions. In order to address this essential issue and enhance the safety and efficiency of connected and autonomous vehicles (CAVs) at multi-lane merging zones, we propose a novel collaborative decision-making framework, named AgentsCoMerge, to leverage large language models (LLMs). Specifically, we first design a scene observation and understanding module to allow an agent to capture the traffic environment. Then we propose a hierarchical planning module to enable the agent to make decisions and plan trajectories based on the observation and the agent's own state. In addition, in order to facilitate collaboration among multiple agents, we introduce a communication module to enable the surrounding agents to exchange necessary information and coordinate their actions. Finally, we develop a reinforcement reflection guided training paradigm to further enhance the decision-making capability of the framework. Extensive experiments are conducted to evaluate the performance of our proposed method, demonstrating its superior efficiency and effectiveness for multi-agent collaborative decision-making under various ramp merging scenarios.

Motivation & Objective

• Address the critical challenge of traffic congestion, safety risks, and inefficiency in multi-lane ramp merging zones.
• Overcome limitations of existing rule-based, probabilistic, and deep learning methods, including poor interpretability, over-reliance on manual discretization, and limited adaptability to diverse driver intentions.
• Develop a decision-making framework that combines high interpretability, minimal manual discretization, and robust adaptability to complex, dynamic traffic scenarios.
• Enable safe and efficient collaborative merging among multiple CAVs through real-time information sharing and coordinated trajectory planning.

Proposed method

• Design a scene observation and understanding module that processes both visual and textual inputs to model the traffic environment comprehensively.
• Implement a hierarchical planning module that generates driving decisions and plans future waypoints based on observed states and agent-specific conditions.
• Introduce an inter-agent communication module enabling CAVs to exchange state information (e.g., position, speed, intent) to coordinate merging order and avoid conflicts.
• Develop a reinforcement reflection guided training paradigm that enhances decision-making by iteratively refining agent behavior through self-evaluation and feedback loops.
• Leverage the reasoning and commonsense capabilities of large language models (LLMs) to support transparent, human-like decision justification and dynamic adaptation.
• Train the framework end-to-end using a combination of imitation learning and reinforcement learning with reflective feedback to improve trajectory accuracy and safety.

Figure 1: The multi-lane merging scenarios

Experimental results

Research questions

• RQ1 Can LLM-powered agents achieve superior safety and efficiency in multi-lane ramp merging compared to traditional rule-based or RL-based methods?
• RQ2 How does inter-agent communication impact the performance and safety of collaborative merging decisions in dense traffic scenarios?
• RQ3 To what extent does reinforcement reflection guided training improve the robustness and accuracy of trajectory prediction and decision-making in complex merging environments?
• RQ4 Can LLM-based frameworks reduce reliance on manual discretization while maintaining high performance and interpretability in autonomous driving decisions?
• RQ5 How do LLMs enable interpretable, human-like reasoning in real-time decision-making under dynamic and uncertain traffic conditions?

Key findings

• The reinforcement reflection guided training reduced the average L2 trajectory error by 90.99% compared to the baseline, demonstrating significant improvement in trajectory prediction accuracy.
• The collision rate was reduced by 97.67% on average with the reinforcement reflection guided training, indicating a major enhancement in safety performance.
• Inter-agent communication improved key metrics: CS increased by 5.75%, ES by 23.82%, SS by 10.51%, and DS by 11.08%, proving its critical role in coordination and safety.
• Case studies confirmed that agents using the framework could reason through complex scenarios, such as yielding to approaching vehicles and merging safely after proper coordination.
• The ego vehicle successfully merged after CAV6 and before CAV7, with surrounding vehicles decelerating to provide space, demonstrating smooth and cooperative traffic flow.
• The framework’s LLM-based reasoning enabled transparent, interpretable decision-making, allowing agents to justify actions using contextual awareness and logical inference.

Figure 2: The kinematic bicycle model of a CAV