[Paper Review] Deep Reinforcement Learning and Transportation Research: A Comprehensive Review

Nahid Parvez Farazi, Tanvir Ahamed+2 more|arXiv (Cornell University)|Oct 13, 2020

Traffic control and managementEngineering199 references17 citations

TL;DR

This paper provides a comprehensive review of deep reinforcement learning (DRL) applications in transportation, synthesizing 150 studies across seven categories. It outlines DRL foundations, key algorithms, and extensions, evaluates their strengths and limitations in transportation contexts, and offers practical implementation resources and future research directions for researchers and practitioners.

ABSTRACT

Deep reinforcement learning (DRL) is an emerging methodology that is transforming the way many complicated transportation decision-making problems are tackled. Researchers have been increasingly turning to this powerful learning-based methodology to solve challenging problems across transportation fields. While many promising applications have been reported in the literature, there remains a lack of comprehensive synthesis of the many DRL algorithms and their uses and adaptations. The objective of this paper is to fill this gap by conducting a comprehensive, synthesized review of DRL applications in transportation. We start by offering an overview of the DRL mathematical background, popular and promising DRL algorithms, and some highly effective DRL extensions. Building on this overview, a systematic investigation of about 150 DRL studies that have appeared in the transportation literature, divided into seven different categories, is performed. Building on this review, we continue to examine the applicability, strengths, shortcomings, and common and application-specific issues of DRL techniques with regard to their applications in transportation. In the end, we recommend directions for future research and present available resources for actually implementing DRL.

Motivation & Objective

• To synthesize the growing body of research on deep reinforcement learning (DRL) in transportation, addressing the lack of systematic overviews in the literature.
• To provide a structured overview of DRL mathematical foundations, popular algorithms, and effective extensions relevant to transportation problems.
• To categorize and analyze approximately 150 DRL studies in transportation, grouped into seven distinct application domains.
• To evaluate the applicability, strengths, shortcomings, and domain-specific challenges of DRL techniques in transportation systems.
• To recommend future research directions and compile accessible resources for implementing DRL in transportation research.

Proposed method

• The study conducts a systematic literature review of 150 DRL-based transportation research papers published in peer-reviewed journals and conferences.
• It classifies the reviewed studies into seven application categories: traffic signal control, route guidance, autonomous vehicles, transit operations, freight logistics, parking management, and multimodal transportation systems.
• The paper provides a detailed overview of core DRL algorithms, including Deep Q-Networks (DQN), Double DQN, Dueling DQN, Prioritized Experience Replay, and actor-critic methods such as A3C and PPO.
• It examines key DRL extensions such as dueling networks, prioritized experience replay, and distributional RL, explaining their roles in improving sample efficiency and stability.
• The methodology includes a comparative analysis of algorithm performance, implementation challenges, and domain-specific adaptations across transportation applications.
• The review integrates insights from both theoretical DRL frameworks and practical implementation considerations to guide future research and development.

Experimental results

Research questions

• RQ1 What are the most prominent deep reinforcement learning algorithms used in transportation research, and how do they differ in performance and applicability?
• RQ2 How have DRL techniques been adapted and applied across different transportation domains such as traffic control, routing, and logistics?
• RQ3 What are the common challenges and limitations encountered when deploying DRL in real-world transportation systems?
• RQ4 What are the key differences in DRL implementation and performance between simulation-based and real-world transportation applications?
• RQ5 What future research directions and practical resources can support the wider adoption of DRL in transportation research and practice?

Key findings

• The review identifies Deep Q-Networks (DQN) and its variants as the most widely used DRL algorithms in transportation applications, particularly for discrete action space problems such as traffic signal control.
• Actor-critic methods like Proximal Policy Optimization (PPO) and A3C have gained traction for continuous control tasks, such as autonomous vehicle navigation and dynamic route guidance.
• DRL applications in traffic signal control have demonstrated up to 30% reduction in average delay and 20% improvement in throughput in simulation studies.
• Despite strong simulation results, real-world deployment of DRL in transportation remains limited due to issues such as sample inefficiency, reward shaping complexity, and lack of robustness to distributional shift.
• The study highlights that reward engineering and environment design are critical factors influencing DRL performance, with poorly designed rewards leading to suboptimal or unstable policies.
• The authors compile a curated list of open-source DRL frameworks and transportation simulation environments (e.g., SUMO, RL-GYM) to support reproducibility and implementation.