[Paper Review] A Survey on Traffic Signal Control Methods

Hua Wei, Guanjie Zheng+2 more|arXiv (Cornell University)|Apr 17, 2019

Traffic control and management65 references177 citations

TL;DR

This survey reviews classical transportation engineering methods and reinforcement learning approaches for traffic signal control, detailing problem formulations, methods, and RL foundations for single and multi-agent settings.

ABSTRACT

Traffic signal control is an important and challenging real-world problem, which aims to minimize the travel time of vehicles by coordinating their movements at the road intersections. Current traffic signal control systems in use still rely heavily on oversimplified information and rule-based methods, although we now have richer data, more computing power and advanced methods to drive the development of intelligent transportation. With the growing interest in intelligent transportation using machine learning methods like reinforcement learning, this survey covers the widely acknowledged transportation approaches and a comprehensive list of recent literature on reinforcement for traffic signal control. We hope this survey can foster interdisciplinary research on this important topic.

Motivation & Objective

• Summarize traditional transportation engineering methods for traffic signal control and their assumptions.
• Highlight limitations of rule-based and optimization-based approaches when facing real-world, dynamic traffic.
• Introduce reinforcement learning foundations for traffic signal control and compare single-agent and multi-agent formulations.
• Provide guidance on data, state representations, and reward design for RL-based traffic signal control.

Proposed method

• Survey classical methods (Webster, GreenWave, Maxband, Actuated, SOTL, Max-pressure, SCATS) with their inputs, outputs, and constraints.
• Describe cycle-based timing, offsets, and bandwidth concepts for coordinated signals.
• Present RL framework for traffic signal control, including MDP, Q-learning, and stochastic games for multi-agent settings.
• Explain how RL integrates state, action, reward, and transition dynamics in isolated and multi-intersection scenarios.

Experimental results

Research questions

• RQ1 What are the traditional optimization-based and rule-based approaches for traffic signal control and their limitations?
• RQ2 How can reinforcement learning be formulated for single-intersection and multi-intersection traffic signal control, and what are key design choices (states, actions, rewards)?
• RQ3 What are the practical data sources and modeling considerations for RL-based traffic signal control?
• RQ4 How do multi-agent RL frameworks (stochastic games) apply to coordinated network traffic signal control?
• RQ5 What benchmarks or comparisons exist between RL-based methods and classical transportation methods?

Key findings

• The survey maps classic methods to their data inputs and outputs, clarifying when fixed-time or actuated strategies are used.
• GreenWave and Maxband illustrate cycle-length constraints and bandwidth-based progression for coordinated signals.
• Actuated and SOTL methods rely on requests and thresholds to adapt phases in real time, while Max-pressure targets network throughput via queue-length pressure balancing.
• SCATS is described as a pre-defined-plan approach with iterative plan selection based on performance metrics.
• RL foundations are presented for single and multi-agent traffic signal control, outlining MDP and stochastic game formulations and the role of rewards.
• The paper highlights the need to integrate richer mobility data and computation power to enable data-driven RL approaches in traffic signal control.