[Paper Review] EMVLight: A Decentralized Reinforcement Learning Framework for Efficient Passage of Emergency Vehicles

Haoran Su, Yaofeng Desmond Zhong+2 more|arXiv (Cornell University)|Sep 12, 2021

Traffic control and management4 citations

TL;DR

EMVLight is a decentralized multi-agent reinforcement learning framework that jointly optimizes dynamic routing and traffic signal control for emergency vehicles (EMVs) in real time. By extending Dijkstra’s algorithm for adaptive route updates and using specialized RL agents to coordinate signal phases, EMVLight reduces EMV travel time by 18% and average non-EMV travel time by 5% compared to state-of-the-art baselines on synthetic and real-world grids.

ABSTRACT

Emergency vehicles (EMVs) play a crucial role in responding to time-critical events such as medical emergencies and fire outbreaks in an urban area. The less time EMVs spend traveling through the traffic, the more likely it would help save people's lives and reduce property loss. To reduce the travel time of EMVs, prior work has used route optimization based on historical traffic-flow data and traffic signal pre-emption based on the optimal route. However, traffic signal pre-emption dynamically changes the traffic flow which, in turn, modifies the optimal route of an EMV. In addition, traffic signal pre-emption practices usually lead to significant disturbances in traffic flow and subsequently increase the travel time for non-EMVs. In this paper, we propose EMVLight, a decentralized reinforcement learning (RL) framework for simultaneous dynamic routing and traffic signal control. EMVLight extends Dijkstra's algorithm to efficiently update the optimal route for the EMVs in real time as it travels through the traffic network. The decentralized RL agents learn network-level cooperative traffic signal phase strategies that not only reduce EMV travel time but also reduce the average travel time of non-EMVs in the network. This benefit has been demonstrated through comprehensive experiments with synthetic and real-world maps. These experiments show that EMVLight outperforms benchmark transportation engineering techniques and existing RL-based signal control methods.

Motivation & Objective

• To address the coupled challenge of dynamic EMV routing and adaptive traffic signal pre-emption under time-varying traffic conditions.
• To reduce EMV travel time without significantly increasing the average travel time of non-EMVs, which is a common drawback of existing pre-emption strategies.
• To design a computationally efficient, real-time routing mechanism that updates optimal paths as EMVs progress through the network.
• To develop a decentralized multi-agent RL framework where agents at intersections cooperatively learn signal phase strategies that balance EMV priority and network-wide efficiency.

Proposed method

• Extends Dijkstra’s algorithm with real-time updates to maintain optimal EMV routes as traffic conditions change during travel.
• Employs a decentralized multi-agent A2C reinforcement learning framework with three agent types: primary pre-emption, secondary pre-emption, and coordination agents.
• Introduces a novel pressure-based reward function that encourages smoothing of vehicle densities in all directions at each intersection, improving network-wide efficiency.
• Uses fingerprinting in policy and value networks to stabilize training and accelerate convergence in multi-agent RL.
• Implements secondary pre-emption agents that learn to 'reserve' road links by selecting signal phases that clear traffic and enable high-speed EMV passage.
• Coordinates signal control across the entire network using local observations and decentralized policies, avoiding centralized coordination bottlenecks.

Experimental results

Research questions

• RQ1 Can a decentralized RL framework simultaneously optimize EMV routing and traffic signal control in a time-dependent traffic environment?
• RQ2 How can EMV travel time be minimized while also reducing the average travel time of non-EMVs, avoiding negative side effects on general traffic flow?
• RQ3 What is the impact of specialized agent types—particularly secondary pre-emption agents—on EMV passage efficiency and network-wide performance?
• RQ4 How does the design of the reward function, especially one based on improved pressure metrics, affect convergence and performance in multi-agent signal control?
• RQ5 To what extent does fingerprinting in the policy and value networks enhance training stability and convergence speed in the EMVLight framework?

Key findings

• EMVLight reduces EMV travel time by 18% on average compared to dynamic routing baselines, demonstrating the effectiveness of its joint routing and signal control approach.
• EMVLight reduces average non-EMV travel time by 5% compared to the best-performing baseline (Max Pressure), showing improved network-wide efficiency.
• Removing secondary pre-emption agents increases EMV travel time by 45%, confirming their critical role in enabling link reservation and smoother EMV passage.
• Replacing the pressure definition with PressLight’s version increases average travel time by 13%, indicating that EMVLight’s improved pressure metric better supports balanced network flow.
• Fingerprinting in the policy and value networks leads to faster convergence and reduced reward fluctuations, validating its role in stabilizing multi-agent RL training.
• On both synthetic and real-world maps, EMVLight outperforms traditional traffic engineering methods and existing RL-based signal control techniques in both EMV travel time and average vehicle delay metrics.