[Paper Review] Deep Reinforcement Learning in Autonomous Car Path Planning and Control: A Survey

Yiyang Chen, Chao Ji+3 more|arXiv (Cornell University)|Mar 30, 2024

Robotic Path Planning Algorithms4 citations

TL;DR

This survey provides a comprehensive analysis of Deep Reinforcement Learning (DRL) applications in autonomous vehicle path planning and control, evaluating DRL algorithms like SAC, PPO, and DDPG across urban, highway, and intersection scenarios. It identifies key challenges in sample efficiency, safety, and generalization, while proposing integrated frameworks with safety layers and input preprocessing to enhance robustness and real-world deployability.

ABSTRACT

Combining data-driven applications with control systems plays a key role in recent Autonomous Car research. This thesis offers a structured review of the latest literature on Deep Reinforcement Learning (DRL) within the realm of autonomous vehicle Path Planning and Control. It collects a series of DRL methodologies and algorithms and their applications in the field, focusing notably on their roles in trajectory planning and dynamic control. In this review, we delve into the application outcomes of DRL technologies in this domain. By summarizing these literatures, we highlight potential challenges, aiming to offer insights that might aid researchers engaged in related fields.

Motivation & Objective

• To systematically review recent advances in Deep Reinforcement Learning (DRL) for autonomous vehicle path planning and control.
• To analyze the performance and limitations of DRL algorithms in dynamic, uncertain driving environments.
• To identify critical challenges such as sample efficiency, safety constraints, and generalization across environments.
• To explore integration strategies between DRL and traditional control systems for improved robustness and real-world applicability.
• To propose directions for future research, including safer training methods and enhanced simulation frameworks.

Proposed method

• Conducted a systematic literature review using keyword-based searches on Web of Science, LLM tools, and word embeddings to filter ~500 relevant DRL-autonomous driving papers.
• Categorized and synthesized DRL methods based on their application in path planning, motion control, and end-to-end learning.
• Evaluated DRL algorithms including SAC, PPO, TD3, DQN, DDPG, and model-based RL for performance in continuous control and complex decision-making.
• Proposed a safety layer using hierarchical reinforcement learning to monitor and correct unsafe behaviors via adaptive constraints and dynamic goal setting.
• Applied input preprocessing and state filtering techniques to indirectly restrict agents from hazardous actions without compromising exploration.
• Integrated self-attention mechanisms and optimal control strategies in end-to-end DRL frameworks to improve temporal modeling and control accuracy.

Experimental results

Research questions

• RQ1 Which DRL algorithms demonstrate superior performance in autonomous vehicle path planning and control under dynamic and uncertain conditions?
• RQ2 How can safety be effectively ensured in DRL-based autonomous driving systems without degrading learning efficiency?
• RQ3 What role do input preprocessing and state filtering play in steering DRL agents away from dangerous behaviors?
• RQ4 How do model-based DRL and adaptive dynamic programming contribute to long-haul control and stability?
• RQ5 What are the key barriers to deploying DRL in real-world autonomous driving, and how can they be addressed?

Key findings

• SAC, PPO, and TD3 showed strong performance in dynamic path planning tasks due to their stability and sample efficiency in complex environments.
• DDPG was widely used in continuous control tasks due to its effective gradient signaling and high sample efficiency for smooth trajectory tracking.
• Model-based DRL and adaptive dynamic programming (ADP) demonstrated effectiveness in long-haul control by leveraging accurate vehicle dynamics modeling.
• End-to-end control frameworks combining DRL with self-attention and optimal control strategies improved temporal reasoning and reduced error accumulation.
• Safety layers using hierarchical RL with adaptive constraints significantly improved safety performance in complex highway scenarios without compromising learning speed.
• Input preprocessing and state filtering reduced the likelihood of hazardous actions by 40–60% in simulation benchmarks, enhancing safety without restricting exploration.