[Paper Review] A Survey on Reinforcement Learning in Aviation Applications

Pouria Razzaghi, Amin Tabrizian+7 more|arXiv (Cornell University)|Nov 3, 2022

Air Traffic Management and Optimization22 citations

TL;DR

A comprehensive survey of reinforcement learning methods and their applications in aviation, covering standard RL formulations, model-based vs model-free approaches, actor-critic and MARL, and selected aviation applications such as collision avoidance, ATFM, ARM, and flight control.

ABSTRACT

Compared with model-based control and optimization methods, reinforcement learning (RL) provides a data-driven, learning-based framework to formulate and solve sequential decision-making problems. The RL framework has become promising due to largely improved data availability and computing power in the aviation industry. Many aviation-based applications can be formulated or treated as sequential decision-making problems. Some of them are offline planning problems, while others need to be solved online and are safety-critical. In this survey paper, we first describe standard RL formulations and solutions. Then we survey the landscape of existing RL-based applications in aviation. Finally, we summarize the paper, identify the technical gaps, and suggest future directions of RL research in aviation.

Motivation & Objective

• Explain RL problem formulation and key concepts in the aviation context.
• Survey model-free RL methods (value-based, policy-based, actor-critic) and their extensions.
• Discuss multi-agent RL and its relevance to aviation applications.
• Review selected RL applications in aviation domains such as collision avoidance, air traffic flow management, revenue management, and flight control.
• Identify gaps and propose future directions for RL research in aviation.

Proposed method

• Present standard RL formulations using MDPs and discuss model-based vs model-free approaches.
• Describe value-based methods (Q-learning, DQN and variants) and their deep extensions.
• Explain policy-based methods (REINFORCE, DPG, DDPG, PPO, TRPO) and actor-critic hybrids.
• Outline multi-agent RL (MARL) frameworks and centralized/decentralized training paradigms.
• Review taxonomy of RL applications in aviation with example studies and approaches.

Experimental results

Research questions

• RQ1 What RL formulations and algorithms are most suitable for aviation sequential decision-making problems?
• RQ2 How have model-based, model-free, and actor-critic methods been applied across aviation domains?
• RQ3 What is the role and effectiveness of MARL in aviation contexts?
• RQ4 What are the key challenges (validation, sim-to-reality, sample efficiency, explainability) hindering real-world RL adoption in aviation?
• RQ5 What future directions can advance RL research for aviation applications?

Key findings

• RL provides a data-driven framework for sequential decision problems in aviation, including safety-critical online decisions.
• A wide range of RL methods—value-based, policy-based, actor-critic, and MARL—have been explored for aviation tasks.
• Collision avoidance, ATFM, ARM, and flight control are active RL application areas with diverse algorithm choices (DQN, PPO, DDPG, MADDPG, SAC).
• Challenges include model validation, simulation-to-reality gap, sample efficiency, and explainability of DRL-based controllers.
• The survey highlights the need for formal validation and flexible, robust RL solutions for real-world aviation deployment.