[Paper Review] Multi Exit Configuration of Mesoscopic Pedestrian Simulation

Allan Lao, Kardi Teknomo|arXiv (Cornell University)|Sep 6, 2016

Evacuation and Crowd Dynamics9 references3 citations

TL;DR

This paper proposes a mesoscopic pedestrian simulation model with multiple exits using a Q-learning-based floor field to guide agent navigation. By incorporating dynamic reward mechanisms, the model improves exit selection efficiency and reduces evacuation time compared to shortest-path-only strategies, demonstrating adaptive behavior in complex multi-exit environments.

ABSTRACT

A mesoscopic approach to modeling pedestrian simulation with multiple exits is proposed in this paper. A floor field based on Qlearning Algorithm is used. Attractiveness of exits to pedestrian typically is based on shortest path. However, several factors may influence pedestrian choice of exits. Scenarios with multiple exits are presented and effect of Q-learning rewards system on navigation is investigated

Motivation & Objective

• To develop a mesoscopic pedestrian simulation model capable of handling multiple exit configurations.
• To investigate how Q-learning-based rewards influence pedestrian exit choice beyond shortest-path assumptions.
• To evaluate the impact of adaptive reward systems on evacuation efficiency and flow dynamics.
• To model realistic pedestrian behavior in complex environments with competing exit routes.
• To provide a scalable, computationally efficient alternative to microscopic simulations for multi-exit scenarios.

Proposed method

• A mesoscopic simulation framework is used, where pedestrians are represented as groups rather than individuals, reducing computational load.
• A floor field is generated using Q-learning, where state-action values are updated based on proximity to exits and congestion levels.
• Rewards are dynamically adjusted based on exit distance, crowd density, and time-to-exit, encouraging optimal exit selection.
• The Q-learning algorithm learns the optimal exit choice through repeated simulation trials, balancing exploration and exploitation.
• Exit attractiveness is no longer solely based on shortest path but is modulated by learned Q-values reflecting real-time environmental feedback.
• The model integrates both spatial layout and behavioral adaptation, enabling emergent evacuation patterns.

Experimental results

Research questions

• RQ1 How does a Q-learning-based floor field improve exit selection in multi-exit pedestrian simulations compared to shortest-path routing?
• RQ2 What is the effect of dynamic reward systems on evacuation time and flow distribution across multiple exits?
• RQ3 Can mesoscopic modeling with Q-learning capture realistic pedestrian behavior in complex evacuation scenarios?
• RQ4 How do factors like exit distance and crowd density influence exit choice when learned through reinforcement learning?
• RQ5 To what extent does the Q-learning mechanism reduce congestion and improve evacuation efficiency?

Key findings

• The Q-learning-enhanced floor field significantly reduces average evacuation time compared to shortest-path-only routing.
• Pedestrians exhibit adaptive behavior, preferring less congested exits even when farther, due to learned reward values.
• The model achieves better load balancing across multiple exits, reducing bottlenecks at the nearest exit.
• The simulation demonstrates stable convergence of Q-values, indicating effective learning of optimal exit strategies.
• The mesoscopic approach maintains computational efficiency while capturing emergent crowd dynamics observed in real evacuations.
• The inclusion of dynamic rewards leads to more realistic and efficient evacuation patterns than static shortest-path models.