[Paper Review] Simulation-to-reality UAV Fault Diagnosis with Deep Learning

Wěi Zhāng, Junjie Tong+2 more|arXiv (Cornell University)|Feb 9, 2023

Target Tracking and Data Fusion in Sensor Networks4 citations

TL;DR

This paper proposes a simulation-to-reality deep learning framework for UAV propeller fault diagnosis using newly identified features (NIF) and domain adaptation (DA) to bridge the sim-to-real gap. By combining NIF with MMD-based domain adaptation and an adjusted simulation model, the method achieves 96% fault detection accuracy on real-world quadrotor flights, marking the first reliable and efficient sim-to-reality fault diagnosis approach for quadrotors.

ABSTRACT

Accurate diagnosis of propeller faults is crucial for ensuring the safe and efficient operation of quadrotors. Training a fault classifier using simulated data and deploying it on a real quadrotor is a cost-effective and safe approach. However, the simulation-to-reality gap often leads to poor performance of the classifier when applied in real flight. In this work, we propose a deep learning model that addresses this issue by utilizing newly identified features (NIF) as input and utilizing domain adaptation techniques to reduce the simulation-to-reality gap. In addition, we introduce an adjusted simulation model that generates training data that more accurately reflects the behavior of real quadrotors. The experimental results demonstrate that our proposed approach achieves an accuracy of 96\% in detecting propeller faults. To the best of our knowledge, this is the first reliable and efficient method for simulation-to-reality fault diagnosis of quadrotor propellers.

Motivation & Objective

• To address the simulation-to-reality gap that degrades fault classifier performance when trained on simulated data and deployed on real UAVs.
• To improve the generalization of deep learning models across domain shifts between simulation and real-world flight data.
• To develop a cost-effective and safe fault diagnosis pipeline that relies on simulated data while maintaining high accuracy in real-world deployment.
• To introduce a novel feature representation (NIF) that enhances model robustness to domain shift and improves learning efficiency.

Proposed method

• Proposes Newly Identified Features (NIF) that exhibit linear relationships between input signals, enabling more efficient and robust deep learning training.
• Employs Maximum Mean Discrepancy (MMD)-based domain adaptation to align the feature distributions of simulated (source) and real (target) domains.
• Designs an adjusted simulation model that better replicates the unbalanced dynamics of real quadrotors, improving the realism of synthetic training data.
• Uses a 1D-CNN (DCNN) architecture with shared layers for both source and target domains, fine-tuned via adversarial domain adaptation.
• Applies t-SNE visualization to confirm that domain-adapted features from simulation and real flight are closely aligned in latent space.
• Trains models using a combination of simulated data (Dataset S) and real flight data (Dataset T), with hyperparameters optimized via early stopping and Adam optimization.

Experimental results

Research questions

• RQ1 Can newly identified features (NIF) improve the generalization of deep learning models across the simulation-to-reality domain shift in UAV fault diagnosis?
• RQ2 To what extent does domain adaptation using MMD reduce the performance gap between simulated and real-world fault classification?
• RQ3 How effective is the proposed method in achieving high accuracy on real flight data when trained exclusively on simulation data?
• RQ4 Can an adjusted simulation model that better reflects real quadrotor dynamics enhance the transferability of the trained classifier?

Key findings

• The DCNN+CF model, using conventional features, achieved only 73% average accuracy on real flight data (Dataset T), highlighting the challenge of sim-to-real transfer.
• The DCNN+NIF model improved accuracy to approximately 80% on real flight data, demonstrating that NIF enhances model generalization across domains.
• The DCNN+NIF+DA model achieved 96% accuracy on real flight data, significantly outperforming both baseline models and proving the effectiveness of domain adaptation.
• t-SNE visualization confirmed that features learned by the DCNN+NIF+DA model are well-aligned between simulated and real flight data, explaining the high transfer performance.
• The method achieved high stability and repeatability, with consistent results across ten independent training runs, indicating robustness.
• To the best of the authors' knowledge, this is the first reliable and efficient method for sim-to-reality fault diagnosis of quadrotor propellers, achieving state-of-the-art accuracy in real-world deployment.