[Paper Review] A Survey on Autonomous Vehicle Control in the Era of Mixed-Autonomy: From Physics-Based to AI-Guided Driving Policy Learning

Xuan Di, Rongye Shi|arXiv (Cornell University)|Jul 10, 2020

Traffic control and managementEngineering244 references18 citations

TL;DR

This survey introduces AI-guided methodologies—particularly deep reinforcement learning, imitation learning, and game theory—for autonomous vehicle (AV) control in mixed-traffic environments spanning human-driven vehicles (HVs) and AVs. It proposes a four-phase deployment framework, identifies key challenges in modeling heterogeneous behaviors and interactions, and outlines a path toward scalable, safe, and ethical AV systems in real-world mixed autonomy ecosystems.

ABSTRACT

This paper serves as an introduction and overview of the potentially useful models and methodologies from artificial intelligence (AI) into the field of transportation engineering for autonomous vehicle (AV) control in the era of mixed autonomy. We will discuss state-of-the-art applications of AI-guided methods, identify opportunities and obstacles, raise open questions, and help suggest the building blocks and areas where AI could play a role in mixed autonomy. We divide the stage of autonomous vehicle (AV) deployment into four phases: the pure HVs, the HV-dominated, the AVdominated, and the pure AVs. This paper is primarily focused on the latter three phases. It is the first-of-its-kind survey paper to comprehensively review literature in both transportation engineering and AI for mixed traffic modeling. Models used for each phase are summarized, encompassing game theory, deep (reinforcement) learning, and imitation learning. While reviewing the methodologies, we primarily focus on the following research questions: (1) What scalable driving policies are to control a large number of AVs in mixed traffic comprised of human drivers and uncontrollable AVs? (2) How do we estimate human driver behaviors? (3) How should the driving behavior of uncontrollable AVs be modeled in the environment? (4) How are the interactions between human drivers and autonomous vehicles characterized? Hopefully this paper will not only inspire our transportation community to rethink the conventional models that are developed in the data-shortage era, but also reach out to other disciplines, in particular robotics and machine learning, to join forces towards creating a safe and efficient mixed traffic ecosystem.

Motivation & Objective

• Address the critical gap in modeling mixed-traffic environments where AVs and human-driven vehicles coexist, especially during transitional deployment phases.
• Overcome limitations of traditional physics-based models in data-scarce environments by integrating AI-driven approaches such as deep reinforcement learning and imitation learning.
• Develop scalable driving policies for large-scale AV deployment in mixed-traffic scenarios involving heterogeneous human drivers and AVs.
• Characterize interactions between human drivers and AVs, including behavioral estimation and modeling of uncontrollable AVs.
• Advance ethical, accountable, and fair AV decision-making through interdisciplinary integration with social science and law.

Proposed method

• Categorize AV deployment into four phases: pure HVs, HV-dominated, AV-dominated, and pure AVs, focusing on the latter three for mixed-autonomy challenges.
• Apply game-theoretic models to analyze strategic interactions between AVs and human drivers in mixed-traffic settings.
• Utilize deep reinforcement learning (DRL) and inverse reinforcement learning (IRL) to learn optimal driving policies from large-scale traffic data.
• Employ imitation learning (IL) to model human driving behavior by learning from demonstrated trajectories, reducing sample bias.
• Integrate prior knowledge from classical traffic models (micro-, meso-, macro-scale) to constrain and improve generalization of AI-based policies.
• Propose multi-scale modeling frameworks to connect microscopic AV behavior with macroscopic traffic system performance.

Experimental results

Research questions

• RQ1 What scalable driving policies can effectively control large numbers of AVs in mixed traffic with human drivers and uncontrollable AVs?
• RQ2 How can human driver behaviors be accurately estimated and modeled under diverse driving conditions and heterogeneous risk profiles?
• RQ3 How should the driving behavior of uncontrollable AVs be represented in mixed-traffic simulation and control frameworks?
• RQ4 How are the complex interactions between human drivers and AVs characterized, especially in terms of safety, efficiency, and emergent traffic dynamics?
• RQ5 How can AI-guided AV control systems be made accountable, fair, and ethically aligned with societal values?

Key findings

• AI-guided methods such as deep reinforcement learning and imitation learning show strong potential for learning scalable, adaptive driving policies in mixed-traffic environments.
• Inverse reinforcement learning (IRL) demonstrates robustness with smaller datasets, offering a promising solution to data scarcity and sample bias in human behavior modeling.
• Existing physics-based models provide valuable prior knowledge that can constrain and improve the generalization of AI-driven policies in mixed-traffic scenarios.
• The HV-dominated and AV-dominated phases present the highest modeling complexity due to unpredictable, heterogeneous interactions between human and autonomous vehicles.
• Field experiment design remains underexplored, with significant uncertainty in how training and test data selection affects model predictive performance and robustness.
• Ethical, accountable, and fair AV decision-making remains an open challenge, requiring interdisciplinary collaboration across engineering, law, and social sciences.