[Paper Review] Intelligent Traffic Signal Control: Using Reinforcement Learning with Partial Detection

Rusheng Zhang, Akihiro Ishikawa+3 more|arXiv (Cornell University)|Jul 4, 2018

Traffic control and management14 citations

TL;DR

This paper proposes a reinforcement learning (RL)-based traffic signal control system for partially detected intelligent transportation systems (ITS), where only vehicles with DSRC-enabled V2I communication are observed. By leveraging RL to handle partial observability and low detection rates, the system significantly reduces average vehicle waiting time even at 20% detection rates, demonstrating robust performance across diverse traffic and network conditions.

ABSTRACT

Intelligent Transportation Systems (ITS) have attracted the attention of researchers and the general public alike as a means to alleviate traffic congestion. Recently, the maturity of wireless technology has enabled a cost-efficient way to achieve ITS by detecting vehicles using Vehicle to Infrastructure (V2I) communications. Traditional ITS algorithms, in most cases, assume that every vehicle is observed, such as by a camera or a loop detector, but a V2I implementation would detect only those vehicles with wireless communications capability. We examine a family of transportation systems, which we will refer to as `Partially Detected Intelligent Transportation Systems'. An algorithm that can act well under a small detection rate is highly desirable due to gradual penetration rates of the underlying wireless technologies such as Dedicated Short Range Communications (DSRC) technology. Artificial Intelligence (AI) techniques for Reinforcement Learning (RL) are suitable tools for finding such an algorithm due to utilizing varied inputs and not requiring explicit analytic understanding or modeling of the underlying system dynamics. In this paper, we report a RL algorithm for partially observable ITS based on DSRC. The performance of this system is studied under different car flows, detection rates, and topologies of the road network. Our system is able to efficiently reduce the average waiting time of vehicles at an intersection, even with a low detection rate.

Motivation & Objective

• Address the challenge of traffic signal control in partially detected ITS, where only a fraction of vehicles are observed due to limited wireless technology penetration.
• Develop an intelligent control algorithm that performs well under low detection rates typical of early-stage DSRC deployment.
• Design a system that does not require full knowledge of traffic dynamics or explicit modeling of the environment.
• Evaluate the performance of the RL-based controller across varying traffic flows, detection rates, and road network topologies.

Proposed method

• Employ reinforcement learning (RL) to learn optimal signal timing policies without requiring explicit modeling of traffic dynamics.
• Design the RL agent to operate under partial observability, using only the state information from DSRC-equipped vehicles.
• Use a value-based RL approach (e.g., Q-learning or DQN-like) to map observed vehicle states to optimal signal phase actions.
• Train the agent using simulated traffic environments with varying detection rates and network topologies.
• Incorporate state representation that aggregates partial vehicle detections into actionable traffic estimates for signal control.
• Optimize the RL reward function to minimize average vehicle delay at intersections.

Experimental results

Research questions

• RQ1 How does the performance of an RL-based traffic signal controller degrade as the detection rate of DSRC-equipped vehicles decreases?
• RQ2 Can an RL agent learn effective signal control policies when only a subset of vehicles is observable?
• RQ3 How does the system perform across different traffic flow levels and road network configurations under partial detection?
• RQ4 What is the minimum detection rate at which the RL controller still achieves meaningful reduction in average vehicle waiting time?

Key findings

• The RL-based controller significantly reduces average vehicle waiting time at intersections, even at low detection rates.
• The system maintains strong performance at detection rates as low as 20%, demonstrating resilience to partial observability.
• The controller outperforms traditional methods under partial detection, where full-observability assumptions fail.
• Performance is robust across diverse traffic conditions and road network topologies.
• The RL approach effectively learns to make optimal signal timing decisions with limited input, without requiring explicit system modeling.