[Paper Review] Adaptive Height Optimisation for Cellular-Connected UAVs using Reinforcement Learning

Erika Fonseca, Boris Galkin+3 more|arXiv (Cornell University)|Jul 27, 2020

UAV Applications and Optimization20 references4 citations

TL;DR

This paper proposes a deep reinforcement learning (DRL)-based approach to dynamically optimize the flight altitude of cellular-connected UAVs to maximize spectral efficiency and throughput. Using Deep Q-Networks (DQN), the UAV learns to adapt its height in real time based on channel conditions, achieving up to a 41% improvement in Quality of Service (QoS) in real-world urban environments, with performance gains dependent on base station density and building height distribution.

ABSTRACT

Providing reliable connectivity to cellular-connected UAV can be very challenging; their performance highly depends on the nature of the surrounding environment, such as density and heights of the ground BSs. On the other hand, tall buildings might block undesired interference signals from ground BSs, thereby improving the connectivity between the UAVs and their serving BSs. To address the connectivity of UAVs in such environments, this paper proposes a RL algorithm to dynamically optimise the height of a UAV as it moves through the environment, with the goal of increasing the throughput or spectrum efficiency that it experiences. The proposed solution is evaluated in two settings: using a series of generated environments where we vary the number of BS and building densities, and in a scenario using real-world data obtained from an experiment in Dublin, Ireland. Results show that our proposed RL-based solution improves UAVs QoS by 6% to 41%, depending on the scenario. We also conclude that, when flying at heights higher than the buildings, building density variation has no impact on UAV QoS. On the other hand, BS density can negatively impact UAV QoS, with higher numbers of BSs generating more interference and deteriorating UAV performance.

Motivation & Objective

• To address the challenge of maintaining reliable cellular connectivity for UAVs in urban environments with high interference and blockage.
• To develop a dynamic, adaptive height control mechanism for UAVs that improves spectral efficiency and throughput without relying on infrastructure redesign.
• To evaluate the proposed RL-based solution in both synthetic environments and real-world measurements from Dublin, Ireland.
• To investigate the impact of base station density and building height on UAV connectivity and optimal flight altitude.
• To demonstrate the feasibility of using real-world experimental data for training and validating UAV altitude optimization policies.

Proposed method

• The UAV's altitude is optimized using a Deep Q-Network (DQN) agent trained to maximize long-term spectral efficiency.
• The state space includes historical SINR values, current and previous altitude, action, and reward signals.
• The action space consists of discrete altitude adjustments (e.g., increase, decrease, or maintain height).
• The reward function is designed to reflect improvements in spectral efficiency and throughput, with penalties for instability or interference.
• The DQN model is trained using experience replay and target networks to stabilize learning.
• The approach is adapted to real-world data by replacing spectral efficiency with measured throughput as the reward signal and omitting neighbor sensing due to data limitations.

Experimental results

Research questions

• RQ1 How does base station density affect the optimal flight altitude for UAVs in urban environments?
• RQ2 How does building density influence UAV connectivity and the effectiveness of altitude adaptation?
• RQ3 Can a DRL-based approach dynamically optimize UAV altitude to improve spectral efficiency and throughput in real-world urban scenarios?
• RQ4 What is the impact of flying above building line-of-sight on interference and QoS?
• RQ5 How does the RL agent's performance compare to baseline strategies like random walk or fixed-height flight?

Key findings

• The proposed DRL-based approach improves UAV Quality of Service (QoS) by 6% to 41% across different scenarios, depending on environmental conditions.
• In high-altitude flight (above building rooftops), building density has no significant impact on UAV QoS, as LoS links dominate.
• Higher base station density increases interference, negatively affecting UAV performance and reducing spectral efficiency.
• The RL agent learns to increase altitude in response to higher BS density, achieving better spectral efficiency than baseline methods.
• The RL agent avoids unnecessary altitude adjustments, making intelligent, energy-efficient decisions rather than moving at every step.
• The model trained on real-world data from Dublin successfully generalizes, demonstrating the feasibility of using empirical measurements for RL-based UAV optimization.